[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:00 dist-test-slave-8hhm krb5kdc[13909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:00 dist-test-slave-8hhm krb5kdc[13909](info): set up 2 sockets
Aug 12 14:47:00 dist-test-slave-8hhm krb5kdc[13909](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 14:47:01.867452 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:01 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:01.996862 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37199
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:37199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:02.109710 13925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:02.110042 13925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:02.110092 13925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:02.111418 13925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:02.111510 13925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:02.111539 13925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:02.111575 13925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:02.111605 13925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:02.113564 13925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:37199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:02.113880 13925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:02.114159 13925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:02.120870 13934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.120863 13936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.120863 13940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:02.120898 13925 server_base.cc:1034] running on GCE node
I20260812 14:47:02.121384 13925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:02.121732 13925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:02.122934 13925 hybrid_clock.cc:648] HybridClock initialized: now 1786546022122917 us; error 83 us; skew 500 ppm
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:02.125967 13925 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:02.126493 13925 webserver.cc:460] Webserver started at http://127.13.146.190:38841/ using document root <none> and password file <none>
I20260812 14:47:02.126793 13925 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:02.126844 13925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:02.127040 13925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:02.128409 13925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "a37075916d7041ff9f01fc66c409f917"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "05bb23d641cad08d6b6bfb8f00872713"
server_key_iv: "a61b21cd83750eb62ded33dbcf2b4abd"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.128831 13925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "a37075916d7041ff9f01fc66c409f917"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "05bb23d641cad08d6b6bfb8f00872713"
server_key_iv: "a61b21cd83750eb62ded33dbcf2b4abd"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.130877 13925 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 14:47:02.131930 13949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.132190 13925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:02.132296 13925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a37075916d7041ff9f01fc66c409f917"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "05bb23d641cad08d6b6bfb8f00872713"
server_key_iv: "a61b21cd83750eb62ded33dbcf2b4abd"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.132377 13925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:02.153739 13925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:02.156956 13925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:02.157251 13925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:02.167735 13925 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:37199
I20260812 14:47:02.167972 14044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:37199 every 8 connection(s)
I20260812 14:47:02.168287 13925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:02.169663 14046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:02.173334 14046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917: Bootstrap starting.
I20260812 14:47:02.173704 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 13925
I20260812 14:47:02.173846 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:02.174903 14046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:02.175014 13898 external_mini_cluster.cc:1428] Setting key 2f9109fc6be0faa74141d1a52aad0d39
I20260812 14:47:02.175872 14046 log.cc:824] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:02.177105 14046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917: No bootstrap required, opened a new log
I20260812 14:47:02.181384 14046 raft_consensus.cc:348] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } }
I20260812 14:47:02.181561 14046 raft_consensus.cc:374] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:02.181622 14046 raft_consensus.cc:729] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37075916d7041ff9f01fc66c409f917, State: Initialized, Role: FOLLOWER
I20260812 14:47:02.181807 14046 consensus_queue.cc:260] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } }
I20260812 14:47:02.181947 14046 raft_consensus.cc:388] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:02.181998 14046 raft_consensus.cc:482] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:02.182045 14046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:02.183286 14046 raft_consensus.cc:504] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } }
I20260812 14:47:02.183414 14046 leader_election.cc:302] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a37075916d7041ff9f01fc66c409f917; no voters:
I20260812 14:47:02.183650 14046 leader_election.cc:288] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:02.183785 14052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:02.183911 14046 sys_catalog.cc:564] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:02.184141 14052 raft_consensus.cc:686] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [term 1 LEADER]: Becoming Leader. State: Replica: a37075916d7041ff9f01fc66c409f917, State: Running, Role: LEADER
I20260812 14:47:02.184257 14052 consensus_queue.cc:237] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } }
I20260812 14:47:02.185693 14055 sys_catalog.cc:455] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a37075916d7041ff9f01fc66c409f917. Latest consensus state: current_term: 1 leader_uuid: "a37075916d7041ff9f01fc66c409f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } } }
I20260812 14:47:02.185807 14055 sys_catalog.cc:458] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:02.186240 14053 sys_catalog.cc:455] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a37075916d7041ff9f01fc66c409f917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a37075916d7041ff9f01fc66c409f917" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37199 } } }
I20260812 14:47:02.186357 14053 sys_catalog.cc:458] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:02.186302 14066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:02.187817 14066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:02.190382 14066 catalog_manager.cc:1347] Generated new cluster ID: 13da5d4d7a234a92bf1f6a41e06b17c1
I20260812 14:47:02.190443 14066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:02.198804 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.176481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.13.146.190:37199)
0812 14:47:02.176828 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.176838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.176907 (+ 69us) server_negotiation.cc:407] Connection header received
0812 14:47:02.179803 (+ 2896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.179818 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.179885 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.180038 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.182305 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.184506 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.186372 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.187121 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.187234 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.193204 (+ 5970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.193208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.193209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.193247 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.196192 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.196755 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.196759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.196759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.196883 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.197241 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.197242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.197243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.197661 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.197987 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.198230 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.198385 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":153,"threads_started":1}
I20260812 14:47:02.208218 14066 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:02.209055 14066 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:02.213474 14066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a37075916d7041ff9f01fc66c409f917: Generated new TSK 0
I20260812 14:47:02.213721 14066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:02.248389 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:02.375769 14083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:02.376044 14083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:02.376088 14083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:02.377395 14083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:02.377475 14083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:02.377523 14083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:02.379194 14083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:02.379518 14083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:02.379783 14083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:02.386197 14095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.386188 14091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.386188 14092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:02.387775 14083 server_base.cc:1034] running on GCE node
I20260812 14:47:02.388027 14083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:02.388269 14083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:02.389417 14083 hybrid_clock.cc:648] HybridClock initialized: now 1786546022389404 us; error 30 us; skew 500 ppm
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:02.392308 14083 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:02.392933 14083 webserver.cc:460] Webserver started at http://127.13.146.129:38977/ using document root <none> and password file <none>
I20260812 14:47:02.393247 14083 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:02.393333 14083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:02.393498 14083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:02.394649 14083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "01d38182c6f743c891839baf87a04890"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "21c37c0f826eb6204cca9e4722a12803"
server_key_iv: "e9a06355ecc500da179d18bc62b795a4"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.395069 14083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "01d38182c6f743c891839baf87a04890"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "21c37c0f826eb6204cca9e4722a12803"
server_key_iv: "e9a06355ecc500da179d18bc62b795a4"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.396610 14083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:02.397773 14106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.397984 14083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:02.398129 14083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "01d38182c6f743c891839baf87a04890"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "21c37c0f826eb6204cca9e4722a12803"
server_key_iv: "e9a06355ecc500da179d18bc62b795a4"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.398217 14083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:02.414893 14083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:02.417339 14083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:02.417529 14083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:02.417796 14083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:02.418223 14083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:02.418284 14083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.418320 14083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:02.418349 14083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.428365 14083 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:42887
I20260812 14:47:02.428529 14324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:42887 every 8 connection(s)
I20260812 14:47:02.428884 14083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:02.438205 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14083
I20260812 14:47:02.438320 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:02.438505 13898 external_mini_cluster.cc:1428] Setting key 0be95625a8449c0a66e0b46d088b0229
I20260812 14:47:02.439967 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.429915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:38973 (local address 127.13.146.190:37199)
0812 14:47:02.429979 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.429981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.430291 (+ 310us) server_negotiation.cc:407] Connection header received
0812 14:47:02.430795 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.430796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.430807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.430864 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.432051 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.432610 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.433846 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.434059 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.434121 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.436721 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.436723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.436723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.436732 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.438290 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.438848 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.438849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.438850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.438926 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.439244 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.439245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.439245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.439381 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.439465 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.439774 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.439875 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:47:02.440184 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.429875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37199 (local address 127.13.146.129:38973)
0812 14:47:02.430141 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:02.430182 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:02.430703 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:02.430924 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:02.430928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:02.431673 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:02.431911 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.431917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.432733 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.432734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.433751 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.433755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.434163 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.434164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.434290 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.434889 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:02.434898 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:02.436606 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:02.438403 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.438412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.438414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.438757 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.439008 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.439009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.439009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.439168 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.439456 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:02.439465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:02.439617 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 14:47:02.439740 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 14:47:02.439920 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":181,"threads_started":1}
I20260812 14:47:02.440744 14326 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37199
I20260812 14:47:02.440856 14326 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:02.441054 14326 heartbeater.cc:511] Master 127.13.146.190:37199 requested a full tablet report, sending...
I20260812 14:47:02.441708 13981 ts_manager.cc:194] Registered new tserver with Master: 01d38182c6f743c891839baf87a04890 (127.13.146.129:42887)
I20260812 14:47:02.442364 13981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:38973
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:02.468178 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:02.555330 14331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:02.555559 14331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:02.555584 14331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:02.556890 14331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:02.556939 14331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:02.556968 14331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:02.558579 14331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:02.558794 14331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:02.558983 14331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:02.564415 14337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.564416 14339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:02.564507 14331 server_base.cc:1034] running on GCE node
W20260812 14:47:02.564455 14341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:02.564855 14331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:02.565140 14331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:02.566313 14331 hybrid_clock.cc:648] HybridClock initialized: now 1786546022566315 us; error 45 us; skew 500 ppm
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:02.568948 14331 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:02.569566 14331 webserver.cc:460] Webserver started at http://127.13.146.130:39305/ using document root <none> and password file <none>
I20260812 14:47:02.569864 14331 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:02.569937 14331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:02.570092 14331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:02.571234 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "10b07488f92a4a41bf957746d5fc0486"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "f65991bb7249dd608fd843c88f80ae99"
server_key_iv: "aea53cadc7a02c605980647b508a0fd1"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.571614 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "10b07488f92a4a41bf957746d5fc0486"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "f65991bb7249dd608fd843c88f80ae99"
server_key_iv: "aea53cadc7a02c605980647b508a0fd1"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.573067 14331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:02.573932 14354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.574116 14331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:02.574236 14331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "10b07488f92a4a41bf957746d5fc0486"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "f65991bb7249dd608fd843c88f80ae99"
server_key_iv: "aea53cadc7a02c605980647b508a0fd1"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.574316 14331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:02.589422 14331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:02.591498 14331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:02.591687 14331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:02.591966 14331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:02.592303 14331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:02.592360 14331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.592398 14331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:02.592444 14331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.599696 14331 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:37657
I20260812 14:47:02.599738 14565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:37657 every 8 connection(s)
I20260812 14:47:02.600093 14331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:02.604461 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14331
I20260812 14:47:02.604568 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:02.604739 13898 external_mini_cluster.cc:1428] Setting key dc73bb915863f74aa5f269e2a5aa84b3
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:02.610293 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.600890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:36315 (local address 127.13.146.190:37199)
0812 14:47:02.600963 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.600965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.601261 (+ 296us) server_negotiation.cc:407] Connection header received
0812 14:47:02.601829 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.601830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.601842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.601893 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.602922 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.603406 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.604429 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.604747 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.604833 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.607194 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.607196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.607196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.607206 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.608631 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.609093 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.609094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.609094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.609166 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.609534 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.609535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.609536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.609697 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.609772 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.610047 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.610150 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:02.610307 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.600854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37199 (local address 127.13.146.130:36315)
0812 14:47:02.601087 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:02.601151 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:02.601727 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:02.601960 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:02.601963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:02.602642 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:02.602843 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.602849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.603488 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.603489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.604324 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.604326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.604847 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.604848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.605027 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.605647 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:02.605656 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:02.607100 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:02.608709 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.608716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.608718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.608975 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.609330 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.609331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.609332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.609455 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.609769 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:02.609774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:02.609893 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 14:47:02.609978 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:47:02.610111 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":129,"threads_started":1}
I20260812 14:47:02.610862 14568 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37199
I20260812 14:47:02.610978 14568 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:02.611200 14568 heartbeater.cc:511] Master 127.13.146.190:37199 requested a full tablet report, sending...
I20260812 14:47:02.611629 13981 ts_manager.cc:194] Registered new tserver with Master: 10b07488f92a4a41bf957746d5fc0486 (127.13.146.130:37657)
I20260812 14:47:02.612018 13981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:36315
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:02.634318 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:02.723373 14574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:02.723588 14574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:02.723613 14574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:02.724781 14574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:02.724815 14574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:02.724841 14574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:02.726415 14574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37199
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:02.726614 14574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:02.726815 14574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:02.732412 14588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.732486 14590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:02.732501 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:02.732667 14574 server_base.cc:1034] running on GCE node
I20260812 14:47:02.732900 14574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:02.733165 14574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:02.734367 14574 hybrid_clock.cc:648] HybridClock initialized: now 1786546022734361 us; error 36 us; skew 500 ppm
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:02.737252 14574 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:02.737825 14574 webserver.cc:460] Webserver started at http://127.13.146.131:46173/ using document root <none> and password file <none>
I20260812 14:47:02.738085 14574 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:02.738157 14574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:02.738317 14574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:02.739427 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "54bf2a2de74f41c389c95887e77194fa"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "164260c5b19bdf7d4ba78857b1743cf0"
server_key_iv: "c0957e1e2c4a8a8b046714afe9a1cde5"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.739826 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "54bf2a2de74f41c389c95887e77194fa"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "164260c5b19bdf7d4ba78857b1743cf0"
server_key_iv: "c0957e1e2c4a8a8b046714afe9a1cde5"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.741374 14574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:02.742177 14603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.742369 14574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:02.742467 14574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "54bf2a2de74f41c389c95887e77194fa"
format_stamp: "Formatted at 2026-08-12 14:47:02 on dist-test-slave-8hhm"
server_key: "164260c5b19bdf7d4ba78857b1743cf0"
server_key_iv: "c0957e1e2c4a8a8b046714afe9a1cde5"
server_key_version: "encryptionkey@0"
I20260812 14:47:02.742555 14574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:02.767889 14574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:02.769932 14574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:02.770113 14574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:02.770381 14574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:02.770718 14574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:02.770776 14574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.770812 14574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:02.770840 14574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:02.778350 14574 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41305
I20260812 14:47:02.778411 14802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41305 every 8 connection(s)
I20260812 14:47:02.778795 14574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:02.781327 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14574
I20260812 14:47:02.781436 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:02.781591 13898 external_mini_cluster.cc:1428] Setting key 3c684aef9bb1f557618da27d9b5e16da
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:02.789474 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.779740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:54527 (local address 127.13.146.190:37199)
0812 14:47:02.779815 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.779817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.780023 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:47:02.780523 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.780524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.780533 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.780596 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.781836 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.782585 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.783718 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.783914 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.783974 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.786344 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.786346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.786346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.786354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.787756 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.788348 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.788349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.788350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.788416 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.788717 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.788717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.788718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.788847 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.788907 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.789294 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.789395 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:47:02.789703 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.779682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37199 (local address 127.13.146.131:54527)
0812 14:47:02.779875 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:02.779922 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:02.780437 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:02.780656 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:02.780659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:02.781398 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:02.781692 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.781701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.782707 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.782707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.783635 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:02.783638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.783995 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.783996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:02.784100 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.784735 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:02.784744 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:02.786249 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:02.787883 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.787893 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.787895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.788238 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.788504 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:02.788505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:02.788505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:02.788644 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:02.788915 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:02.788918 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:02.789064 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:47:02.789167 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 14:47:02.789418 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260812 14:47:02.790452 14803 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37199
I20260812 14:47:02.790601 14803 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:02.790845 14803 heartbeater.cc:511] Master 127.13.146.190:37199 requested a full tablet report, sending...
I20260812 14:47:02.791342 13981 ts_manager.cc:194] Registered new tserver with Master: 54bf2a2de74f41c389c95887e77194fa (127.13.146.131:41305)
I20260812 14:47:02.791779 13981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:54527
I20260812 14:47:02.793921 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:02.820013 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.811528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.13.146.129:42887)
0812 14:47:02.811758 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.811763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.811775 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:02.811805 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.811807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.811821 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.811895 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.812516 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.813382 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.814102 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.814443 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.814508 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.816893 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.816906 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.816906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.816914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.818737 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.819139 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.819142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.819142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.819205 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.819432 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.819433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.819433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.819585 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.819713 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.819812 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.819883 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":128,"threads_started":1}
W20260812 14:47:02.820492 14164 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54110
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[13909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:02.846099 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:02.838777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.13.146.129:42887)
0812 14:47:02.838850 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:02.838853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:02.838912 (+ 59us) server_negotiation.cc:407] Connection header received
0812 14:47:02.838966 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:02.838967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:02.838981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:02.839031 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:02.839667 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.840208 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.840954 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:02.841158 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:02.841309 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:02.843578 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:02.843579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:02.843579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:02.843586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:02.845046 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.845467 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.845468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.845468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.845525 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:02.845682 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:02.845683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:02.845683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:02.845782 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:02.845873 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:02.845939 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 14:47:02.846007 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:02.846752 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14083
I20260812 14:47:02.866818 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14331
I20260812 14:47:02.891963 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14574
I20260812 14:47:02.916613 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 13925
2026-08-12T14:47:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2413 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[14829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[14829](info): set up 2 sockets
Aug 12 14:47:02 dist-test-slave-8hhm krb5kdc[14829](info): commencing operation
krb5kdc: starting...
W20260812 14:47:04.235704 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:04.364005 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45595
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:04.449671 14854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:04.449898 14854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:04.449921 14854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:04.451012 14854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:04.451047 14854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:04.451061 14854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:04.451073 14854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:04.451085 14854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:04.452539 14854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:04.452752 14854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:04.452930 14854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:04.458235 14870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:04.458396 14869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:04.458396 14854 server_base.cc:1034] running on GCE node
W20260812 14:47:04.458469 14873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:04.458763 14854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:04.459077 14854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:04.460269 14854 hybrid_clock.cc:648] HybridClock initialized: now 1786546024460241 us; error 43 us; skew 500 ppm
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:04.468449 14854 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:04.468935 14854 webserver.cc:460] Webserver started at http://127.13.146.190:43357/ using document root <none> and password file <none>
I20260812 14:47:04.469201 14854 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:04.469306 14854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:04.469512 14854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:04.470690 14854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "1912b23cef7f40428c69c273cae3e7a4"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "b51f3473eb373b4d82bb25c1d789abdb"
server_key_iv: "8ce9377699941f9d73ec0113a93aa57f"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.471098 14854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "1912b23cef7f40428c69c273cae3e7a4"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "b51f3473eb373b4d82bb25c1d789abdb"
server_key_iv: "8ce9377699941f9d73ec0113a93aa57f"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.472633 14854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:04.473760 14886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.473973 14854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:04.474071 14854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "1912b23cef7f40428c69c273cae3e7a4"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "b51f3473eb373b4d82bb25c1d789abdb"
server_key_iv: "8ce9377699941f9d73ec0113a93aa57f"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.474133 14854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:04.515934 14854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:04.518450 14854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:04.518574 14854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:04.522866 14854 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:45595
I20260812 14:47:04.522903 14987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:45595 every 8 connection(s)
I20260812 14:47:04.523275 14854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:04.524312 14991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:04.526772 14991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4: Bootstrap starting.
I20260812 14:47:04.527355 14991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:04.527614 14991 log.cc:824] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:04.528348 14991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4: No bootstrap required, opened a new log
I20260812 14:47:04.529989 14991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } }
I20260812 14:47:04.530133 14991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:04.530156 14991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1912b23cef7f40428c69c273cae3e7a4, State: Initialized, Role: FOLLOWER
I20260812 14:47:04.530315 14991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } }
I20260812 14:47:04.530413 14991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:04.530438 14991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:04.530519 14991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:04.530519 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14854
I20260812 14:47:04.530606 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:04.530788 13898 external_mini_cluster.cc:1428] Setting key 9f351e59c11d1167a8910febfda381f1
I20260812 14:47:04.531275 14991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } }
I20260812 14:47:04.531432 14991 leader_election.cc:302] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1912b23cef7f40428c69c273cae3e7a4; no voters:
I20260812 14:47:04.531641 14991 leader_election.cc:288] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:04.531672 14999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:04.531859 14999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [term 1 LEADER]: Becoming Leader. State: Replica: 1912b23cef7f40428c69c273cae3e7a4, State: Running, Role: LEADER
I20260812 14:47:04.531914 14991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:04.531996 14999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } }
I20260812 14:47:04.532593 15002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1912b23cef7f40428c69c273cae3e7a4. Latest consensus state: current_term: 1 leader_uuid: "1912b23cef7f40428c69c273cae3e7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } } }
I20260812 14:47:04.532704 15002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:04.532994 15016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:04.533574 15016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:04.534683 15003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1912b23cef7f40428c69c273cae3e7a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1912b23cef7f40428c69c273cae3e7a4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45595 } } }
I20260812 14:47:04.534832 15003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:04.535319 15016 catalog_manager.cc:1347] Generated new cluster ID: 06d8ad4fe0a64628954315704b2fe950
I20260812 14:47:04.535379 15016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:04.543099 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:04.531310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.13.146.190:45595)
0812 14:47:04.531515 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:04.531529 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:04.531592 (+ 63us) server_negotiation.cc:407] Connection header received
0812 14:47:04.531763 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:04.531773 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:04.531837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:04.532075 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:04.532912 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.534371 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.535432 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.536083 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.536222 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:04.539040 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:04.539044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:04.539045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:04.539058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:04.541012 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.541466 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.541470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.541470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.541600 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.541802 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.541802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.541802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.542104 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:04.542251 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:04.542724 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 14:47:04.542867 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260812 14:47:04.546512 15016 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:04.547024 15016 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:04.551018 15016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1912b23cef7f40428c69c273cae3e7a4: Generated new TSK 0
I20260812 14:47:04.551352 15016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:04.583990 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:04.669868 15038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:04.670083 15038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:04.670109 15038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:04.671265 15038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:04.671310 15038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:04.671363 15038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:04.672957 15038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:04.673223 15038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:04.673493 15038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:04.678859 15045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:04.678840 15048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:04.678819 15044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:04.679157 15038 server_base.cc:1034] running on GCE node
I20260812 14:47:04.679458 15038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:04.679721 15038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:04.680894 15038 hybrid_clock.cc:648] HybridClock initialized: now 1786546024680879 us; error 27 us; skew 500 ppm
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:04.683753 15038 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:04.684226 15038 webserver.cc:460] Webserver started at http://127.13.146.129:36393/ using document root <none> and password file <none>
I20260812 14:47:04.684478 15038 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:04.684549 15038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:04.684712 15038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:04.685899 15038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "e593d2d0914f1b19d27179f450afb9e3"
server_key_iv: "666bd5a4f847c0c4704011c52e54ff58"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.686299 15038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "e593d2d0914f1b19d27179f450afb9e3"
server_key_iv: "666bd5a4f847c0c4704011c52e54ff58"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.687721 15038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:04.688473 15061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.688673 15038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:04.688787 15038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "e593d2d0914f1b19d27179f450afb9e3"
server_key_iv: "666bd5a4f847c0c4704011c52e54ff58"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.688869 15038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:04.701124 15038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:04.703164 15038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:04.703356 15038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:04.703620 15038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:04.703998 15038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:04.704059 15038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.704110 15038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:04.704140 15038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.710505 15038 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44571
I20260812 14:47:04.710553 15273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44571 every 8 connection(s)
I20260812 14:47:04.710911 15038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:04.720249 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15038
I20260812 14:47:04.720342 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:04.720541 13898 external_mini_cluster.cc:1428] Setting key cfb9f8fabb653133f85b53de7a8593c9
I20260812 14:47:04.721771 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:04.711794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:48633 (local address 127.13.146.190:45595)
0812 14:47:04.711874 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:04.711877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:04.712102 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:47:04.713455 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:04.713457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:04.713475 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:04.713532 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:04.714234 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.714768 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.715795 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.716022 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.716096 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:04.718534 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:04.718536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:04.718536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:04.718545 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:04.720075 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.720566 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.720567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.720568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.720648 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.720986 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.720987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.720988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.721149 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:04.721269 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:04.721627 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 14:47:04.721706 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:47:04.722103 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:04.711773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45595 (local address 127.13.146.129:48633)
0812 14:47:04.711968 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:04.712009 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:04.713325 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:04.713602 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:04.713605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:04.713898 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:04.714110 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:04.714114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.714867 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.714868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:04.715698 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:04.715700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.716143 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.716144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:04.716255 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:04.716808 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:04.716817 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:04.718424 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:04.720177 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:04.720185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:04.720187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:04.720493 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:04.720747 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:04.720748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:04.720748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:04.720908 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:04.721263 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:04.721270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:04.721459 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 14:47:04.721601 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 14:47:04.721798 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260812 14:47:04.722715 15275 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45595
I20260812 14:47:04.722831 15275 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:04.723047 15275 heartbeater.cc:511] Master 127.13.146.190:45595 requested a full tablet report, sending...
I20260812 14:47:04.723730 14915 ts_manager.cc:194] Registered new tserver with Master: ad58d08e6c2e4c8bba8398ce4fc5a4e3 (127.13.146.129:44571)
I20260812 14:47:04.724665 14915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:48633
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:04.748238 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:04.835454 15283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:04.835677 15283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:04.835704 15283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:04.836941 15283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:04.837047 15283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:04.837080 15283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:04.838737 15283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:04.839011 15283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:04.839349 15283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:04.845481 15299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:04.845484 15295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:04.845481 15296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:04.845777 15283 server_base.cc:1034] running on GCE node
I20260812 14:47:04.846043 15283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:04.846266 15283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:04.847410 15283 hybrid_clock.cc:648] HybridClock initialized: now 1786546024847395 us; error 41 us; skew 500 ppm
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:04.850163 15283 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:04.850666 15283 webserver.cc:460] Webserver started at http://127.13.146.130:41329/ using document root <none> and password file <none>
I20260812 14:47:04.850880 15283 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:04.850924 15283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:04.851013 15283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:04.852084 15283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "9b35d2c617a74eb286037089a0f182fd"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "43354fa76d746884a0226886adc822aa"
server_key_iv: "993fd54f88e41db194b9adaf0d536d1c"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.852406 15283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "9b35d2c617a74eb286037089a0f182fd"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "43354fa76d746884a0226886adc822aa"
server_key_iv: "993fd54f88e41db194b9adaf0d536d1c"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.853845 15283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:04.854930 15309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.855139 15283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:47:04.855230 15283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "9b35d2c617a74eb286037089a0f182fd"
format_stamp: "Formatted at 2026-08-12 14:47:04 on dist-test-slave-8hhm"
server_key: "43354fa76d746884a0226886adc822aa"
server_key_iv: "993fd54f88e41db194b9adaf0d536d1c"
server_key_version: "encryptionkey@0"
I20260812 14:47:04.855295 15283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:04.875126 15283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:04.877707 15283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:04.877831 15283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:04.878023 15283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:04.878360 15283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:04.878396 15283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.878453 15283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:04.878480 15283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:04.886224 15283 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:42291
I20260812 14:47:04.886328 15499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:42291 every 8 connection(s)
I20260812 14:47:04.886741 15283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:04.894718 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15283
I20260812 14:47:04.894802 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:04.894999 13898 external_mini_cluster.cc:1428] Setting key 691f658d475e42ae8a0842ac87e20880
Aug 12 14:47:04 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:04.899402 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:04.888120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:49805 (local address 127.13.146.190:45595)
0812 14:47:04.888200 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:04.888202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:04.888298 (+ 96us) server_negotiation.cc:407] Connection header received
0812 14:47:04.889906 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:04.889908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:04.889922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:04.889970 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:04.890898 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.891934 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.893427 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.893686 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.893783 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:04.896405 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:04.896407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:04.896408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:04.896418 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:04.897870 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.898356 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.898358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.898358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.898428 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:04.898717 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:04.898718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:04.898718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:04.898841 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:04.898922 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:04.899226 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 14:47:04.899314 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:47:04.899612 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:04.887771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45595 (local address 127.13.146.130:49805)
0812 14:47:04.888001 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:04.888191 (+ 190us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:04.889792 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:04.890069 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:04.890074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:04.890439 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:04.890759 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:04.890766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.892092 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.892093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:04.893335 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:04.893338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:04.893796 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:04.893798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:04.893955 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:04.894457 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:04.894466 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:04.896243 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:04.897992 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:04.897999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:04.898002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:04.898245 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:04.898504 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:04.898505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:04.898505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:04.898640 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:04.898912 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:04.898920 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:04.899068 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 14:47:04.899178 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:47:04.899353 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":123,"threads_started":1}
I20260812 14:47:04.900101 15501 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45595
I20260812 14:47:04.900218 15501 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:04.900401 15501 heartbeater.cc:511] Master 127.13.146.190:45595 requested a full tablet report, sending...
I20260812 14:47:04.900866 14915 ts_manager.cc:194] Registered new tserver with Master: 9b35d2c617a74eb286037089a0f182fd (127.13.146.130:42291)
I20260812 14:47:04.901435 14915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:49805
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:04.924413 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:05.010347 15513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:05.010564 15513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:05.010589 15513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:05.012298 15513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:05.012348 15513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:05.012377 15513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:05.013886 15513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:05.014097 15513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:05.014282 15513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:05.019565 15527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:05.019588 15520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:05.019582 15522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:05.019912 15513 server_base.cc:1034] running on GCE node
I20260812 14:47:05.020186 15513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:05.020476 15513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:05.021639 15513 hybrid_clock.cc:648] HybridClock initialized: now 1786546025021619 us; error 31 us; skew 500 ppm
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:05.024434 15513 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:05.024941 15513 webserver.cc:460] Webserver started at http://127.13.146.131:35089/ using document root <none> and password file <none>
I20260812 14:47:05.025185 15513 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:05.025286 15513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:05.025461 15513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:05.026580 15513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "0e53a3e8a7f24b65b61241cbada55140"
format_stamp: "Formatted at 2026-08-12 14:47:05 on dist-test-slave-8hhm"
server_key: "a8a24c76d47cb0fe8281c6d1e17fee07"
server_key_iv: "cb0a33a57d28ef03dc9bae22a61421bf"
server_key_version: "encryptionkey@0"
I20260812 14:47:05.026981 15513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "0e53a3e8a7f24b65b61241cbada55140"
format_stamp: "Formatted at 2026-08-12 14:47:05 on dist-test-slave-8hhm"
server_key: "a8a24c76d47cb0fe8281c6d1e17fee07"
server_key_iv: "cb0a33a57d28ef03dc9bae22a61421bf"
server_key_version: "encryptionkey@0"
I20260812 14:47:05.028415 15513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:05.029289 15542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:05.029476 15513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:05.029587 15513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "0e53a3e8a7f24b65b61241cbada55140"
format_stamp: "Formatted at 2026-08-12 14:47:05 on dist-test-slave-8hhm"
server_key: "a8a24c76d47cb0fe8281c6d1e17fee07"
server_key_iv: "cb0a33a57d28ef03dc9bae22a61421bf"
server_key_version: "encryptionkey@0"
I20260812 14:47:05.029675 15513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:05.046608 15513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:05.049080 15513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:05.049286 15513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:05.049552 15513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:05.049891 15513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:05.049947 15513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:05.049994 15513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:05.050034 15513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:05.056398 15513 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36491
I20260812 14:47:05.056445 15732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36491 every 8 connection(s)
I20260812 14:47:05.056771 15513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:05.060532 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15513
I20260812 14:47:05.060618 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:05.060794 13898 external_mini_cluster.cc:1428] Setting key 8288665cfe569ad4a8abecfbcb55c42d
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:05.067364 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.057572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:56697 (local address 127.13.146.190:45595)
0812 14:47:05.057707 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.057709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.057915 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:47:05.059045 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.059047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.059060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.059114 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:05.059840 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.060383 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.061968 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.062170 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.062228 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.064457 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:05.064458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:05.064459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:05.064466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:05.066061 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.066504 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.066505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.066505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.066562 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.066808 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.066808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.066809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.066917 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:05.066970 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.067223 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.067296 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 14:47:05.067683 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.057541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45595 (local address 127.13.146.131:56697)
0812 14:47:05.057769 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.057820 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.058922 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.059185 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.059189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.059544 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:05.059742 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.059747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.060486 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.060487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.061886 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.061895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.062244 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.062245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.062348 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.062863 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:05.062872 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:05.064364 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:05.066154 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:05.066161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:05.066163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:05.066412 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:05.066632 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:05.066633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:05.066633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:05.066749 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:05.066971 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:05.066976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:05.067107 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.067194 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.067324 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260812 14:47:05.068261 15734 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45595
I20260812 14:47:05.068363 15734 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:05.068531 15734 heartbeater.cc:511] Master 127.13.146.190:45595 requested a full tablet report, sending...
I20260812 14:47:05.068894 14915 ts_manager.cc:194] Registered new tserver with Master: 0e53a3e8a7f24b65b61241cbada55140 (127.13.146.131:36491)
I20260812 14:47:05.069388 14915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:56697
I20260812 14:47:05.072741 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:47:05.081033 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.074472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.13.146.190:45595)
0812 14:47:05.074584 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.074587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.074750 (+ 163us) server_negotiation.cc:407] Connection header received
0812 14:47:05.074807 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.074808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.074817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.074869 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:05.075487 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.075978 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.076788 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.076983 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.077042 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.078454 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:05.078455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:05.078456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:05.078462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:05.079938 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.080290 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.080291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.080291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.080345 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.080574 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.080574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.080575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.080722 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:05.080823 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.080901 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.080963 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:47:05.083635 14915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:05.085299 14915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:05.092423 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.087784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36491 (local address 127.0.0.1:52560)
0812 14:47:05.088025 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.088094 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.088159 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.088488 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.088492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.088495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:05.088655 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.088658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.090232 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.090233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.091260 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.091261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.092095 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.092177 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092191 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.092263 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.092326 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":53,"threads_started":1}
I20260812 14:47:05.092526 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.088069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.13.146.130:42291)
0812 14:47:05.088584 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.088589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.088599 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:05.088638 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.088640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.088655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.088888 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:05.088969 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.090144 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.091387 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.092138 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092237 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092286 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.092394 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.092436 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":85,"threads_started":1}
I20260812 14:47:05.092648 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.088341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.13.146.129:44571)
0812 14:47:05.088535 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.088539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.088560 (+ 21us) server_negotiation.cc:407] Connection header received
0812 14:47:05.088598 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.088600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.088614 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.088689 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:05.088943 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.089760 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.091333 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.092103 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092184 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092248 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.092493 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.092556 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260812 14:47:05.092548 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.087833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42291 (local address 127.0.0.1:33856)
0812 14:47:05.088046 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.088084 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.088159 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.088757 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.088758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.088760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:05.088815 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.088816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.090269 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.090270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.091301 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.091305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.092243 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.092300 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092303 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.092377 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.092410 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260812 14:47:05.092794 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.087833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44571 (local address 127.0.0.1:49892)
0812 14:47:05.088278 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.088288 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.088329 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.088803 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.088804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.088806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:05.088859 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.088862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.089863 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.089864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.091233 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.091235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.092285 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.092350 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092356 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.092407 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.092451 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260812 14:47:05.093468 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.088131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.13.146.131:36491)
0812 14:47:05.088296 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.088300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.088311 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:05.088359 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.088361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.088374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.088480 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:05.088752 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.090114 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.091326 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.091974 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.092124 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.092192 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.093154 (+ 962us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.093387 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260812 14:47:05.093879 15149 tablet_service.cc:1467] Processing CreateTablet for tablet e5265c0c405d4adcb64a482b05f6a041 (DEFAULT_TABLE table=test-table [id=54ae9ab33bf841179d215bc060977c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:05.093902 15379 tablet_service.cc:1467] Processing CreateTablet for tablet e5265c0c405d4adcb64a482b05f6a041 (DEFAULT_TABLE table=test-table [id=54ae9ab33bf841179d215bc060977c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:05.094195 15149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5265c0c405d4adcb64a482b05f6a041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:05.094256 15379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5265c0c405d4adcb64a482b05f6a041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:05.094163 15622 tablet_service.cc:1467] Processing CreateTablet for tablet e5265c0c405d4adcb64a482b05f6a041 (DEFAULT_TABLE table=test-table [id=54ae9ab33bf841179d215bc060977c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:05.094493 15622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5265c0c405d4adcb64a482b05f6a041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:05.096711 15777 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Bootstrap starting.
I20260812 14:47:05.097456 15777 tablet_bootstrap.cc:654] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:05.097784 15779 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Bootstrap starting.
I20260812 14:47:05.097937 15778 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Bootstrap starting.
I20260812 14:47:05.098161 15777 log.cc:824] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:05.098615 15779 tablet_bootstrap.cc:654] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:05.098613 15778 tablet_bootstrap.cc:654] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:05.099067 15777 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: No bootstrap required, opened a new log
I20260812 14:47:05.099164 15777 ts_tablet_manager.cc:1397] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:47:05.099146 15779 log.cc:824] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:05.099421 15778 log.cc:824] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:05.099970 15779 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: No bootstrap required, opened a new log
I20260812 14:47:05.100065 15779 ts_tablet_manager.cc:1397] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:05.100770 15777 raft_consensus.cc:348] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.100929 15777 raft_consensus.cc:374] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:05.100951 15777 raft_consensus.cc:729] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad58d08e6c2e4c8bba8398ce4fc5a4e3, State: Initialized, Role: FOLLOWER
I20260812 14:47:05.101120 15777 consensus_queue.cc:260] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.101468 15777 ts_tablet_manager.cc:1428] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:47:05.101516 15275 heartbeater.cc:503] Master 127.13.146.190:45595 was elected leader, sending a full tablet report...
I20260812 14:47:05.101663 15779 raft_consensus.cc:348] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.101845 15779 raft_consensus.cc:374] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:05.101893 15779 raft_consensus.cc:729] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e53a3e8a7f24b65b61241cbada55140, State: Initialized, Role: FOLLOWER
I20260812 14:47:05.102008 15779 consensus_queue.cc:260] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.102126 15778 tablet_bootstrap.cc:492] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: No bootstrap required, opened a new log
I20260812 14:47:05.102195 15778 ts_tablet_manager.cc:1397] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 14:47:05.102298 15779 ts_tablet_manager.cc:1428] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:47:05.102411 15734 heartbeater.cc:503] Master 127.13.146.190:45595 was elected leader, sending a full tablet report...
I20260812 14:47:05.104017 15778 raft_consensus.cc:348] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.104188 15778 raft_consensus.cc:374] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:05.104236 15778 raft_consensus.cc:729] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b35d2c617a74eb286037089a0f182fd, State: Initialized, Role: FOLLOWER
I20260812 14:47:05.104354 15778 consensus_queue.cc:260] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.104631 15778 ts_tablet_manager.cc:1428] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:05.104717 15501 heartbeater.cc:503] Master 127.13.146.190:45595 was elected leader, sending a full tablet report...
W20260812 14:47:05.137457 15504 tablet.cc:2369] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:05.203739 15786 raft_consensus.cc:482] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:05.203878 15786 raft_consensus.cc:504] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.204329 15786 leader_election.cc:288] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b35d2c617a74eb286037089a0f182fd (127.13.146.130:42291), ad58d08e6c2e4c8bba8398ce4fc5a4e3 (127.13.146.129:44571)
I20260812 14:47:05.208297 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.204678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:60173 (local address 127.13.146.130:42291)
0812 14:47:05.204724 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.204726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.204825 (+ 99us) server_negotiation.cc:407] Connection header received
0812 14:47:05.204899 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.204900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.204909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.204963 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:05.205113 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.205896 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.207481 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.207901 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.207996 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.208006 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.208180 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.208220 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:05.208324 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.204655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:48121 (local address 127.13.146.129:44571)
0812 14:47:05.204706 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.204709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.204722 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:05.204818 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.204819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.204831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.204896 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:05.205114 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.205896 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.207528 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.207930 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.208023 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.208032 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.208214 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.208254 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:47:05.208333 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.204560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44571 (local address 127.13.146.131:48121)
0812 14:47:05.204652 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.204666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.204717 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.204914 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.204915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.204917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:05.205010 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.205012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.205995 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.205996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.207397 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.207400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.208026 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.208027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.208123 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.208126 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.208190 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.208235 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":94}
I20260812 14:47:05.208355 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.204571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42291 (local address 127.13.146.131:60173)
0812 14:47:05.204753 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:05.204763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:05.204823 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:05.204982 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:05.204982 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:05.204984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:05.205041 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.205042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.205995 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.205996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.207397 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:05.207400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.207996 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.207998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:05.208083 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.208086 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:05.208167 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:47:05.208210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":76,"thread_start_us":106,"threads_started":1}
I20260812 14:47:05.208706 15427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5265c0c405d4adcb64a482b05f6a041" candidate_uuid: "0e53a3e8a7f24b65b61241cbada55140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b35d2c617a74eb286037089a0f182fd" is_pre_election: true
I20260812 14:47:05.208705 15180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5265c0c405d4adcb64a482b05f6a041" candidate_uuid: "0e53a3e8a7f24b65b61241cbada55140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" is_pre_election: true
I20260812 14:47:05.208881 15180 raft_consensus.cc:2454] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e53a3e8a7f24b65b61241cbada55140 in term 0.
I20260812 14:47:05.208881 15427 raft_consensus.cc:2454] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e53a3e8a7f24b65b61241cbada55140 in term 0.
I20260812 14:47:05.209139 15543 leader_election.cc:302] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e53a3e8a7f24b65b61241cbada55140, ad58d08e6c2e4c8bba8398ce4fc5a4e3; no voters:
I20260812 14:47:05.209365 15786 raft_consensus.cc:2781] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:05.209408 15786 raft_consensus.cc:482] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:05.209424 15786 raft_consensus.cc:3037] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:05.210310 15786 raft_consensus.cc:504] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.210466 15786 leader_election.cc:288] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [CANDIDATE]: Term 1 election: Requested vote from peers 9b35d2c617a74eb286037089a0f182fd (127.13.146.130:42291), ad58d08e6c2e4c8bba8398ce4fc5a4e3 (127.13.146.129:44571)
I20260812 14:47:05.210662 15427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5265c0c405d4adcb64a482b05f6a041" candidate_uuid: "0e53a3e8a7f24b65b61241cbada55140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b35d2c617a74eb286037089a0f182fd"
I20260812 14:47:05.210668 15180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5265c0c405d4adcb64a482b05f6a041" candidate_uuid: "0e53a3e8a7f24b65b61241cbada55140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3"
I20260812 14:47:05.210747 15427 raft_consensus.cc:3037] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:05.210777 15180 raft_consensus.cc:3037] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:05.211513 15427 raft_consensus.cc:2454] T e5265c0c405d4adcb64a482b05f6a041 P 9b35d2c617a74eb286037089a0f182fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e53a3e8a7f24b65b61241cbada55140 in term 1.
I20260812 14:47:05.211705 15180 raft_consensus.cc:2454] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e53a3e8a7f24b65b61241cbada55140 in term 1.
I20260812 14:47:05.211705 15545 leader_election.cc:302] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e53a3e8a7f24b65b61241cbada55140, 9b35d2c617a74eb286037089a0f182fd; no voters:
I20260812 14:47:05.211836 15786 raft_consensus.cc:2781] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 1 FOLLOWER]: Leader election won for term 1
W20260812 14:47:05.211874 15277 tablet.cc:2369] T e5265c0c405d4adcb64a482b05f6a041 P ad58d08e6c2e4c8bba8398ce4fc5a4e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:05.212008 15786 raft_consensus.cc:686] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [term 1 LEADER]: Becoming Leader. State: Replica: 0e53a3e8a7f24b65b61241cbada55140, State: Running, Role: LEADER
I20260812 14:47:05.212116 15786 consensus_queue.cc:237] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } }
I20260812 14:47:05.213089 14912 catalog_manager.cc:5495] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e53a3e8a7f24b65b61241cbada55140 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "0e53a3e8a7f24b65b61241cbada55140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e53a3e8a7f24b65b61241cbada55140" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b35d2c617a74eb286037089a0f182fd" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad58d08e6c2e4c8bba8398ce4fc5a4e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44571 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:05.245082 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.237550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.13.146.129:44571)
0812 14:47:05.237640 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.237642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.237786 (+ 144us) server_negotiation.cc:407] Connection header received
0812 14:47:05.237836 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.237837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.237848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.237899 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:05.238463 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.238950 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.239666 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.239858 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.239908 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.242281 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:05.242294 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:05.242294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:05.242302 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:05.243869 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.244276 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.244279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.244280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.244345 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.244584 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.244584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.244585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.244724 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:05.244830 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.244916 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.244982 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:05.271579 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.263742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.13.146.129:44571)
0812 14:47:05.263829 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.263831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.263925 (+ 94us) server_negotiation.cc:407] Connection header received
0812 14:47:05.264029 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.264030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.264042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.264120 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:05.264736 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.265315 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.266083 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.266299 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.266379 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.268803 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:05.268804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:05.268805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:05.268813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:05.270376 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.270740 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.270741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.270741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.270798 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.271030 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.271030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.271030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.271125 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:05.271204 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.271409 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.271478 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[14829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:05.297701 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:05.289899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.13.146.129:44571)
0812 14:47:05.289975 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:05.289977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:05.290102 (+ 125us) server_negotiation.cc:407] Connection header received
0812 14:47:05.290196 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:05.290197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:05.290216 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:05.290283 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:05.290892 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.291391 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.292174 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:05.292363 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:05.292441 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:05.294888 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:05.294890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:05.294891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:05.294899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:05.296343 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.296827 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.296828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.296829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.296889 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:05.297113 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:05.297114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:05.297114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:05.297276 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:05.297394 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:05.297506 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:47:05.297594 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 14:47:05.298228 15115 tablet_service.cc:2908] Rejecting scan request for tablet e5265c0c405d4adcb64a482b05f6a041: Uninitialized: clean time has not yet been initialized
I20260812 14:47:05.299266 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15038
W20260812 14:47:05.309000 15737 tablet.cc:2369] T e5265c0c405d4adcb64a482b05f6a041 P 0e53a3e8a7f24b65b61241cbada55140: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:05.324499 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15283
I20260812 14:47:05.335278 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15513
I20260812 14:47:05.349470 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14854
2026-08-12T14:47:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2436 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[15817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[15817](info): set up 2 sockets
Aug 12 14:47:05 dist-test-slave-8hhm krb5kdc[15817](info): commencing operation
krb5kdc: starting...
W20260812 14:47:06.744119 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:06 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:06.876956 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:42935
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:42935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:06.972890 15836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:06.973151 15836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:06.973181 15836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:06.974521 15836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:06.974573 15836 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:47:06.974591 15836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:06.974606 15836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:06.974619 15836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:06.974681 15836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:06.976233 15836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:42935
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:42935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:06.976533 15836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:06.976788 15836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:06.981974 15849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:06.982016 15854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:06.982024 15852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:06.982254 15836 server_base.cc:1034] running on GCE node
I20260812 14:47:06.982553 15836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:06.982861 15836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:06.984033 15836 hybrid_clock.cc:648] HybridClock initialized: now 1786546026984027 us; error 52 us; skew 500 ppm
Aug 12 14:47:06 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546026, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:06.986680 15836 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:06.987061 15836 webserver.cc:460] Webserver started at http://127.13.146.190:37851/ using document root <none> and password file <none>
I20260812 14:47:06.987316 15836 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:06.987363 15836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:06.987516 15836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:06.988598 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "7d77bad5d05e479b80054fb30abe0743"
format_stamp: "Formatted at 2026-08-12 14:47:06 on dist-test-slave-8hhm"
server_key: "3b88350dd772a895dbbdcd0e6a787a5f"
server_key_iv: "a355f5159297808c54e489c2fe231898"
server_key_version: "encryptionkey@0"
I20260812 14:47:06.989015 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "7d77bad5d05e479b80054fb30abe0743"
format_stamp: "Formatted at 2026-08-12 14:47:06 on dist-test-slave-8hhm"
server_key: "3b88350dd772a895dbbdcd0e6a787a5f"
server_key_iv: "a355f5159297808c54e489c2fe231898"
server_key_version: "encryptionkey@0"
I20260812 14:47:06.990554 15836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:06.991374 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:06.991560 15836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:06.991684 15836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "7d77bad5d05e479b80054fb30abe0743"
format_stamp: "Formatted at 2026-08-12 14:47:06 on dist-test-slave-8hhm"
server_key: "3b88350dd772a895dbbdcd0e6a787a5f"
server_key_iv: "a355f5159297808c54e489c2fe231898"
server_key_version: "encryptionkey@0"
I20260812 14:47:06.991796 15836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:07.002472 15836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:07.004833 15836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:07.005004 15836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:07.010088 15836 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:42935
I20260812 14:47:07.010185 15977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:42935 every 8 connection(s)
I20260812 14:47:07.010532 15836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:07.011427 15979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.012775 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15836
I20260812 14:47:07.012861 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:07.013038 13898 external_mini_cluster.cc:1428] Setting key 11a21f27fd5882bff197e72440525075
I20260812 14:47:07.014418 15979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Bootstrap starting.
I20260812 14:47:07.015111 15979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.015440 15979 log.cc:824] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:07.016319 15979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: No bootstrap required, opened a new log
I20260812 14:47:07.018666 15979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } }
I20260812 14:47:07.018838 15979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.018910 15979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d77bad5d05e479b80054fb30abe0743, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.019091 15979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } }
I20260812 14:47:07.019201 15979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:07.019236 15979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:07.019279 15979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:07.020020 15979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } }
I20260812 14:47:07.020159 15979 leader_election.cc:302] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d77bad5d05e479b80054fb30abe0743; no voters:
I20260812 14:47:07.020371 15979 leader_election.cc:288] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:07.020403 15985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:07.020529 15985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [term 1 LEADER]: Becoming Leader. State: Replica: 7d77bad5d05e479b80054fb30abe0743, State: Running, Role: LEADER
I20260812 14:47:07.020624 15985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } }
I20260812 14:47:07.020658 15979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:07.021330 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d77bad5d05e479b80054fb30abe0743. Latest consensus state: current_term: 1 leader_uuid: "7d77bad5d05e479b80054fb30abe0743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } } }
I20260812 14:47:07.021525 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:07.021337 15986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d77bad5d05e479b80054fb30abe0743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d77bad5d05e479b80054fb30abe0743" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 42935 } } }
I20260812 14:47:07.021773 15986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:07.022287 16004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:07.022801 16004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:07.024760 16004 catalog_manager.cc:1347] Generated new cluster ID: 24f41fd3a6dd4345a5a0989f39ba274b
I20260812 14:47:07.024829 16004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:07.024981 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.013647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.13.146.190:42935)
0812 14:47:07.013849 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.013857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.013884 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:47:07.014298 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.014304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.014350 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.014501 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.015153 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.016310 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.017055 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.017569 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.017718 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.020572 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.020577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.020578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.020592 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.023180 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.023611 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.023614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.023614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.023694 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.023898 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.023899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.023900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.024222 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.024386 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.024541 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.024690 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260812 14:47:07.036594 16004 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:07.037176 16004 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:07.045267 16004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Generated new TSK 0
I20260812 14:47:07.045471 16004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:07.078588 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:42935
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:07.179240 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:07.179464 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:07.179500 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:07.180733 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:07.180783 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:07.180805 16021 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:07.180862 16021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:07.182437 16021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:42935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:07.182722 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:07.182974 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:07.188474 16036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:07.188503 16034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:07.188480 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:07.188753 16021 server_base.cc:1034] running on GCE node
I20260812 14:47:07.188990 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:07.189308 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:07.190495 16021 hybrid_clock.cc:648] HybridClock initialized: now 1786546027190465 us; error 44 us; skew 500 ppm
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:07.193264 16021 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:07.193792 16021 webserver.cc:460] Webserver started at http://127.13.146.129:43469/ using document root <none> and password file <none>
I20260812 14:47:07.194067 16021 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:07.194141 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:07.194309 16021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:07.195616 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "956b9e2b87ea4a09815eb6e250701049"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "2cbfd7583e30af1e88b59c2b968d2a69"
server_key_iv: "8b8dc463e7d70741e9abc1f3b3e618fc"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.196054 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "956b9e2b87ea4a09815eb6e250701049"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "2cbfd7583e30af1e88b59c2b968d2a69"
server_key_iv: "8b8dc463e7d70741e9abc1f3b3e618fc"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.197535 16021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 14:47:07.198390 16053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.198567 16021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:07.198689 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "956b9e2b87ea4a09815eb6e250701049"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "2cbfd7583e30af1e88b59c2b968d2a69"
server_key_iv: "8b8dc463e7d70741e9abc1f3b3e618fc"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.198765 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:07.215348 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:07.217554 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:07.217774 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:07.218401 16021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:07.218465 16021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.218508 16021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:07.218529 16021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:07.226691 16021 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:42643
I20260812 14:47:07.226789 16247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:42643 every 8 connection(s)
I20260812 14:47:07.227124 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:07.229758 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.218829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:58653 (local address 127.13.146.190:42935)
0812 14:47:07.218904 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.218906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.219145 (+ 239us) server_negotiation.cc:407] Connection header received
0812 14:47:07.219689 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.219691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.219708 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.219777 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.220603 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.221200 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.222574 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.222828 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.222920 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.226794 (+ 3874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.226796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.226796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.226805 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.228420 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.228857 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.228858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.228859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.228929 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.229167 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.229168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.229168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.229335 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.229392 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.229613 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.229690 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:47:07.229969 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.218815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.129:58653)
0812 14:47:07.218993 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.219036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.219562 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.219889 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.219894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.220234 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.220480 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.220488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.221350 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.221351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.222468 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.222471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.222937 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.222938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.223072 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.223719 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.223733 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.226644 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.228540 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.228545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.228546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.228780 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.228993 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.228994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.228994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.229097 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.229384 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.229387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.229505 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.229592 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.229723 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260812 14:47:07.230512 16248 heartbeater.cc:347] Connected to a master server at 127.13.146.190:42935
I20260812 14:47:07.230613 16248 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:07.230816 16248 heartbeater.cc:511] Master 127.13.146.190:42935 requested a full tablet report, sending...
I20260812 14:47:07.231365 15890 ts_manager.cc:194] Registered new tserver with Master: 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643)
I20260812 14:47:07.232100 15890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:58653
I20260812 14:47:07.235599 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16021
I20260812 14:47:07.235689 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:07.235890 13898 external_mini_cluster.cc:1428] Setting key 0695fd72141a8534a29fb601bca70043
I20260812 14:47:07.237277 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.231661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:60915 (local address 127.13.146.190:42935)
0812 14:47:07.231708 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.231710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.231790 (+ 80us) server_negotiation.cc:407] Connection header received
0812 14:47:07.231862 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.231863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.231873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.231922 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.232250 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.232734 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.233523 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.233727 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.233803 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.234680 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.234681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.234682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.234689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.235948 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.236320 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.236322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.236322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.236385 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.236683 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.236684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.236684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.236802 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.236887 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.237062 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.237147 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:07.237283 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.231597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.129:60915)
0812 14:47:07.231718 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.231729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.231805 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.231924 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.231924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.232114 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.232171 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.232173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.232833 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.232834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.233432 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.233434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.233802 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.233804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.233880 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.234266 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.234270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.234572 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.236013 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.236015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.236015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.236240 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.236455 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.236456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.236456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.236522 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.236879 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.236880 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.236942 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.237031 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.237146 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:07.263998 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:42935
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:07.348805 16261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:07.349017 16261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:07.349043 16261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:07.350303 16261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:07.350351 16261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:07.350373 16261 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:07.350391 16261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:07.351938 16261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:42935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:07.352192 16261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:07.352444 16261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:07.357797 16269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:07.357851 16261 server_base.cc:1034] running on GCE node
W20260812 14:47:07.357832 16270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:07.357832 16274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:07.358337 16261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:07.358596 16261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:07.359763 16261 hybrid_clock.cc:648] HybridClock initialized: now 1786546027359750 us; error 29 us; skew 500 ppm
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:07.362632 16261 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:07.363122 16261 webserver.cc:460] Webserver started at http://127.13.146.130:40997/ using document root <none> and password file <none>
I20260812 14:47:07.363389 16261 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:07.363435 16261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:07.363582 16261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:07.364873 16261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "4ad0c22644984ead8d621351b9445f23"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "0fcbc77889c67790cebeb389e8252d0a"
server_key_iv: "fc6aa219961868f03a094bf65ff4a893"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.365280 16261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "4ad0c22644984ead8d621351b9445f23"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "0fcbc77889c67790cebeb389e8252d0a"
server_key_iv: "fc6aa219961868f03a094bf65ff4a893"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.366755 16261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:07.367607 16288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.367789 16261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:07.367903 16261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "4ad0c22644984ead8d621351b9445f23"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "0fcbc77889c67790cebeb389e8252d0a"
server_key_iv: "fc6aa219961868f03a094bf65ff4a893"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.367980 16261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:07.394980 16261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:07.397594 16261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:07.397801 16261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:07.398487 16261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:07.398546 16261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.398581 16261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:07.398602 16261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.406369 16261 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:43301
I20260812 14:47:07.406456 16489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:43301 every 8 connection(s)
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:07.406798 16261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:07.410075 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.399020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:38993 (local address 127.13.146.190:42935)
0812 14:47:07.399103 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.399106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.399394 (+ 288us) server_negotiation.cc:407] Connection header received
0812 14:47:07.400036 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.400038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.400061 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.400159 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.401238 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.402006 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.403803 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.404038 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.404156 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.407091 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.407093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.407094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.407105 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.408498 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.408972 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.408973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.408974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.409045 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.409370 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.409371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.409371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.409510 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.409592 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.409886 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.409979 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:47:07.410276 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16261
I20260812 14:47:07.410305 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.399033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.130:38993)
0812 14:47:07.399240 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.399285 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.399828 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.400287 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.400292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.400737 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.401073 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.401081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.402125 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.402126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.403677 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.403682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.404177 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.404178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.404345 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.405069 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.405079 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.406961 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.408633 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.408641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.408643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.408884 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.409117 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.409118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.409119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.409293 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.409583 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.409590 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.409740 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.409851 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.410058 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":126,"threads_started":1}
I20260812 14:47:07.410367 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:07.410557 13898 external_mini_cluster.cc:1428] Setting key 25e1ed52a3ec5dbae49499a3c20f0720
I20260812 14:47:07.411302 16491 heartbeater.cc:347] Connected to a master server at 127.13.146.190:42935
I20260812 14:47:07.411449 16491 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:07.411667 16491 heartbeater.cc:511] Master 127.13.146.190:42935 requested a full tablet report, sending...
I20260812 14:47:07.412106 15892 ts_manager.cc:194] Registered new tserver with Master: 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301)
I20260812 14:47:07.412478 15892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:38993
I20260812 14:47:07.419040 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.412051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57009 (local address 127.13.146.190:42935)
0812 14:47:07.412088 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.412090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.413368 (+ 1278us) server_negotiation.cc:407] Connection header received
0812 14:47:07.413412 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.413413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.413422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.413463 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.413857 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.414310 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.415001 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.415194 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.415271 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.416312 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.416313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.416314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.416320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.417777 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.418168 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.418170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.418170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.418234 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.418440 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.418441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.418441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.418597 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.418685 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.418862 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.418944 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:47:07.419059 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.411883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.130:57009)
0812 14:47:07.413280 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.413294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.413364 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.413469 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.413470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.413698 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.413785 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.413787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.414408 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.414409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.414915 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.414916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.415271 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.415272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.415346 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.415869 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.415874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.416200 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.417888 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.417889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.417890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.418069 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.418305 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.418305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.418306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.418366 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.418676 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.418677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.418740 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.418829 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.418942 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:07.437922 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:42935
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:07.523561 16505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:07.523769 16505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:07.523794 16505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:07.525110 16505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:07.525154 16505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:07.525175 16505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:07.525192 16505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:07.526674 16505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:42935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:07.526868 16505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:07.527051 16505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:07.532327 16516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:07.532320 16522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:07.532423 16505 server_base.cc:1034] running on GCE node
W20260812 14:47:07.532320 16519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:07.532783 16505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:07.533080 16505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:07.534230 16505 hybrid_clock.cc:648] HybridClock initialized: now 1786546027534231 us; error 42 us; skew 500 ppm
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:07.536829 16505 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:07.537389 16505 webserver.cc:460] Webserver started at http://127.13.146.131:43833/ using document root <none> and password file <none>
I20260812 14:47:07.537664 16505 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:07.537711 16505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:07.537906 16505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:07.539005 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "d12b5321e86f49f2ae7c027dc242d07a"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "02d8a3142edb4b68f210f8821309b55f"
server_key_iv: "13f8cca5c020ba9723035194eb9981ad"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.539393 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "d12b5321e86f49f2ae7c027dc242d07a"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "02d8a3142edb4b68f210f8821309b55f"
server_key_iv: "13f8cca5c020ba9723035194eb9981ad"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.540783 16505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:07.541651 16533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.541832 16505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:07.541939 16505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "d12b5321e86f49f2ae7c027dc242d07a"
format_stamp: "Formatted at 2026-08-12 14:47:07 on dist-test-slave-8hhm"
server_key: "02d8a3142edb4b68f210f8821309b55f"
server_key_iv: "13f8cca5c020ba9723035194eb9981ad"
server_key_version: "encryptionkey@0"
I20260812 14:47:07.542022 16505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:07.559386 16505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:07.561590 16505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:07.561785 16505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:07.562389 16505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:07.562463 16505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.562515 16505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:07.562546 16505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:07.569906 16505 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36551
I20260812 14:47:07.570144 16748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36551 every 8 connection(s)
I20260812 14:47:07.570415 16505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:07.573622 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16505
I20260812 14:47:07.573735 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:07.573913 13898 external_mini_cluster.cc:1428] Setting key 28f2893e04f16142d83ad2a839239f75
I20260812 14:47:07.574903 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.563053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:40763 (local address 127.13.146.190:42935)
0812 14:47:07.563128 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.563131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.563429 (+ 298us) server_negotiation.cc:407] Connection header received
0812 14:47:07.564145 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.564147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.564161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.564340 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.565384 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.566253 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.568024 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.568376 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.568482 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.571496 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.571497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.571498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.571506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.573343 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.573768 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.573770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.573771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.573842 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.574183 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.574184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.574185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.574338 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.574428 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.574718 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.574810 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 14:47:07.575145 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.563033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.131:40763)
0812 14:47:07.563266 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.563306 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.563968 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.564454 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.564460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.564904 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.565265 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.565274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.566370 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.566372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.567917 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.567919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.568505 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.568506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.568692 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.569501 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.569510 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.571387 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.573422 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.573429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.573430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.573678 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.573913 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.573914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.573915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.574071 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.574419 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.574426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.574576 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.574695 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.574865 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":133,"threads_started":1}
I20260812 14:47:07.575706 16751 heartbeater.cc:347] Connected to a master server at 127.13.146.190:42935
I20260812 14:47:07.575804 16751 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:07.576004 16751 heartbeater.cc:511] Master 127.13.146.190:42935 requested a full tablet report, sending...
I20260812 14:47:07.576375 15892 ts_manager.cc:194] Registered new tserver with Master: d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131:36551)
I20260812 14:47:07.576844 15892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:40763
I20260812 14:47:07.582327 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.576853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:39733 (local address 127.13.146.190:42935)
0812 14:47:07.576922 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.576924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.577097 (+ 173us) server_negotiation.cc:407] Connection header received
0812 14:47:07.577173 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.577174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.577184 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.577268 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.577605 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.578078 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.578779 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.578987 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.579063 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.579960 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.579961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.579962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.579969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.581087 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.581503 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.581504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.581504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.581568 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.581776 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.581777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.581778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.581901 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.581976 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.582160 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.582235 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:47:07.582362 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.576810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.13.146.131:39733)
0812 14:47:07.577024 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.577034 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.577113 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.577291 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.577291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.577465 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:07.577525 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.577527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.578175 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.578177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.578692 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.578694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.579062 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.579063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.579138 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.579544 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:07.579548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:07.579865 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:07.581240 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.581242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.581242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.581424 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.581639 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:07.581640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:07.581640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:07.581703 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:07.581979 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:07.581981 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:07.582043 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.582127 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.582236 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260812 14:47:07.585675 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:47:07 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:07.610505 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.603183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.13.146.190:42935)
0812 14:47:07.603257 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.603258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.603268 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:07.603297 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.603298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.603309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.603377 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:07.603952 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.604470 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.605307 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.605502 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.605566 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.607903 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:07.607904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:07.607904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:07.607911 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:07.609479 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.609905 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.609906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.609906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.609960 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:07.610123 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:07.610123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:07.610123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:07.610224 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:07.610297 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.610381 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.610439 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:47:07.612308 15892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:07.614080 15892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:07.621147 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.13.146.129:42643)
0812 14:47:07.617089 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.617093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.617101 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:07.617135 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.617137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.617149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.617268 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:07.617424 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618595 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.619834 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.620630 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620785 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.620851 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.620996 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.621035 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":87,"threads_started":1}
I20260812 14:47:07.621181 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36551 (local address 127.0.0.1:43092)
0812 14:47:07.616931 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.616939 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.616980 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.617324 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.617325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.617327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:07.617385 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.617387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618645 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.618646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.619919 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.619922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.620801 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.620872 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.620890 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.620971 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.621038 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":95,"thread_start_us":77,"threads_started":1}
I20260812 14:47:07.621181 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:42643 (local address 127.0.0.1:57224)
0812 14:47:07.616696 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.616744 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.616812 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.617270 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.617273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.617275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:07.617348 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.617350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618700 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.618701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.619738 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.619740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.620784 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.620872 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.620890 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.620971 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.621036 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":63,"thread_start_us":67,"threads_started":1}
I20260812 14:47:07.621877 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.13.146.131:36551)
0812 14:47:07.617122 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.617127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.617155 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:47:07.617197 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.617199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.617232 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.617308 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:07.617470 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618551 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620011 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.620733 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620805 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.620851 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.621449 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.621489 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260812 14:47:07.622160 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:43301 (local address 127.0.0.1:56322)
0812 14:47:07.616778 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.616789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.616842 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.617639 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.617639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.617642 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:07.617683 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.617684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618549 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.618549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.619919 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.619922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.621915 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.621916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.621993 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.621996 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.622045 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.622087 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":59,"thread_start_us":95,"threads_started":1}
I20260812 14:47:07.622316 16142 tablet_service.cc:1467] Processing CreateTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (DEFAULT_TABLE table=test-table [id=cbf5c5f3e27c464590c547dbfb001337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.622725 16142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.622706 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (DEFAULT_TABLE table=test-table [id=cbf5c5f3e27c464590c547dbfb001337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.623047 16624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.625494 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.616643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.13.146.130:43301)
0812 14:47:07.617407 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.617411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.617421 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:07.617460 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.617462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.617478 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.617552 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:07.617745 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.618444 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.620893 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.621737 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.621822 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.621878 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.625325 (+ 3447us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.625392 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":91,"threads_started":1}
I20260812 14:47:07.626005 16802 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Bootstrap starting.
I20260812 14:47:07.626207 16386 tablet_service.cc:1467] Processing CreateTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (DEFAULT_TABLE table=test-table [id=cbf5c5f3e27c464590c547dbfb001337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.626469 16386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.626646 16802 tablet_bootstrap.cc:654] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.626892 16802 log.cc:824] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:07.627226 16801 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Bootstrap starting.
I20260812 14:47:07.627665 16802 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: No bootstrap required, opened a new log
I20260812 14:47:07.627728 16802 ts_tablet_manager.cc:1397] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:07.628259 16801 tablet_bootstrap.cc:654] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.628584 16801 log.cc:824] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:07.629438 16807 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Bootstrap starting.
I20260812 14:47:07.629470 16801 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: No bootstrap required, opened a new log
I20260812 14:47:07.629532 16801 ts_tablet_manager.cc:1397] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:07.629886 16802 raft_consensus.cc:348] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.630071 16802 raft_consensus.cc:374] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.630117 16802 raft_consensus.cc:729] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956b9e2b87ea4a09815eb6e250701049, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.630136 16807 tablet_bootstrap.cc:654] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.630270 16802 consensus_queue.cc:260] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.630458 16807 log.cc:824] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:07.630589 16802 ts_tablet_manager.cc:1428] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:07.630659 16248 heartbeater.cc:503] Master 127.13.146.190:42935 was elected leader, sending a full tablet report...
I20260812 14:47:07.631240 16807 tablet_bootstrap.cc:492] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: No bootstrap required, opened a new log
I20260812 14:47:07.631314 16807 ts_tablet_manager.cc:1397] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:07.631704 16801 raft_consensus.cc:348] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.631896 16801 raft_consensus.cc:374] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.631939 16801 raft_consensus.cc:729] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12b5321e86f49f2ae7c027dc242d07a, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.632061 16801 consensus_queue.cc:260] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.632328 16801 ts_tablet_manager.cc:1428] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:07.632372 16751 heartbeater.cc:503] Master 127.13.146.190:42935 was elected leader, sending a full tablet report...
I20260812 14:47:07.633412 16807 raft_consensus.cc:348] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.633574 16807 raft_consensus.cc:374] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.633600 16807 raft_consensus.cc:729] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad0c22644984ead8d621351b9445f23, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.633708 16807 consensus_queue.cc:260] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.633940 16807 ts_tablet_manager.cc:1428] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:47:07.634013 16491 heartbeater.cc:503] Master 127.13.146.190:42935 was elected leader, sending a full tablet report...
W20260812 14:47:07.657531 16492 tablet.cc:2369] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:47:07.727988 16249 tablet.cc:2369] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:47:07.821529 16754 tablet.cc:2369] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:07.919308 16816 raft_consensus.cc:482] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:07.919450 16816 raft_consensus.cc:504] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.919906 16816 leader_election.cc:288] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643), 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301)
I20260812 14:47:07.923318 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.920205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:44995 (local address 127.13.146.129:42643)
0812 14:47:07.920265 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.920267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.920281 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:07.920365 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.920367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.920378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.920476 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:07.920680 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.921313 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.922486 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.922916 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.922973 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.922980 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.923189 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.923237 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:47:07.923352 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.920245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:38269 (local address 127.13.146.130:43301)
0812 14:47:07.920291 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.920293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.920488 (+ 195us) server_negotiation.cc:407] Connection header received
0812 14:47:07.920555 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.920556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.920568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.920651 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:07.920853 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.921470 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.922598 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.922966 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.923041 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.923049 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.923238 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.923284 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:47:07.923352 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.920135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:42643 (local address 127.13.146.131:44995)
0812 14:47:07.920213 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.920228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.920285 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.920496 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.920497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.920499 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:07.920595 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.920597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.921382 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.921384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.922377 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.922380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.922995 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.922996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.923082 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.923087 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.923164 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.923223 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":27}
I20260812 14:47:07.923359 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.920143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:43301 (local address 127.13.146.131:38269)
0812 14:47:07.920428 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:07.920439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:07.920498 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:07.920662 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:07.920669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:07.920671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:07.920747 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.920749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.921584 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.921585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.922517 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:07.922519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.923066 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.923067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:07.923137 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.923140 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:07.923217 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:47:07.923253 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":68,"thread_start_us":123,"threads_started":1}
I20260812 14:47:07.923771 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a2e8d61f6745a5b45a2ac2c8ea05ce" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23" is_pre_election: true
I20260812 14:47:07.923797 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a2e8d61f6745a5b45a2ac2c8ea05ce" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956b9e2b87ea4a09815eb6e250701049" is_pre_election: true
I20260812 14:47:07.923942 16180 raft_consensus.cc:2454] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 0.
I20260812 14:47:07.923943 16419 raft_consensus.cc:2454] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 0.
I20260812 14:47:07.924182 16536 leader_election.cc:302] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad0c22644984ead8d621351b9445f23, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:07.924324 16816 raft_consensus.cc:2781] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:07.924383 16816 raft_consensus.cc:482] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:07.924407 16816 raft_consensus.cc:3037] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:07.925717 16816 raft_consensus.cc:504] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.925875 16816 leader_election.cc:288] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 election: Requested vote from peers 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643), 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301)
I20260812 14:47:07.926056 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a2e8d61f6745a5b45a2ac2c8ea05ce" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23"
I20260812 14:47:07.926045 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04a2e8d61f6745a5b45a2ac2c8ea05ce" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956b9e2b87ea4a09815eb6e250701049"
I20260812 14:47:07.926136 16180 raft_consensus.cc:3037] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:07.926136 16419 raft_consensus.cc:3037] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:07.927028 16419 raft_consensus.cc:2454] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 1.
I20260812 14:47:07.927028 16180 raft_consensus.cc:2454] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 1.
I20260812 14:47:07.927228 16539 leader_election.cc:302] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956b9e2b87ea4a09815eb6e250701049, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:07.927345 16816 raft_consensus.cc:2781] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:07.927491 16816 raft_consensus.cc:686] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 1 LEADER]: Becoming Leader. State: Replica: d12b5321e86f49f2ae7c027dc242d07a, State: Running, Role: LEADER
I20260812 14:47:07.927603 16816 consensus_queue.cc:237] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.928475 15892 catalog_manager.cc:5495] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a reported cstate change: term changed from 0 to 1, leader changed from <none> to d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131). New cstate: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:07.975131 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:07.972470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43100 (local address 127.13.146.131:36551)
0812 14:47:07.972553 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:07.972556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:07.972567 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:07.972609 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:07.972610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:07.972637 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:07.972712 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:07.972867 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.973394 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.974209 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:07.974411 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:07.974492 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:07.974705 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:07.974812 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:07.974907 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:07.974953 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 14:47:07.974978 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 14:47:07.977042 16180 raft_consensus.cc:1261] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Refusing update from remote peer d12b5321e86f49f2ae7c027dc242d07a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:07.977051 16419 raft_consensus.cc:1261] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Refusing update from remote peer d12b5321e86f49f2ae7c027dc242d07a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:07.977458 16820 consensus_queue.cc:1035] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Connected to new peer: Peer: permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:07.977605 16816 consensus_queue.cc:1035] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:07.979967 16827 mvcc.cc:205] Tried to move back new op lower bound from 7317692530591178752 to 7317692530392170496. Current Snapshot: MvccSnapshot[applied={T|T < 7317692530591178752}]
I20260812 14:47:07.980612 16828 mvcc.cc:205] Tried to move back new op lower bound from 7317692530591178752 to 7317692530392170496. Current Snapshot: MvccSnapshot[applied={T|T < 7317692530591178752}]
I20260812 14:47:07.981369 16831 mvcc.cc:205] Tried to move back new op lower bound from 7317692530591178752 to 7317692530392170496. Current Snapshot: MvccSnapshot[applied={T|T < 7317692530591178752 or (T in {7317692530591178752})}]
I20260812 14:47:07.984061 15892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:07.984223 15892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:07.985162 15892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Sending DeleteTablet for 3 replicas of tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce
I20260812 14:47:07.985692 16142 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57224
I20260812 14:47:07.985687 16624 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:43092
I20260812 14:47:07.985746 16386 tablet_service.cc:1514] Processing DeleteTablet for tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56322
I20260812 14:47:07.985919 16839 tablet_replica.cc:330] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: stopping tablet replica
I20260812 14:47:07.986028 16839 raft_consensus.cc:2229] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:07.986057 15892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:47:07.986153 16839 raft_consensus.cc:2258] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 14:47:07.986219 15892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:07.986366 16841 tablet_replica.cc:330] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: stopping tablet replica
I20260812 14:47:07.986465 16841 raft_consensus.cc:2229] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:07.986588 16841 raft_consensus.cc:2258] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:07.986624 16842 tablet_replica.cc:330] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: stopping tablet replica
I20260812 14:47:07.987058 16839 ts_tablet_manager.cc:1905] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:07.987361 16842 raft_consensus.cc:2229] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:07.987484 16842 raft_consensus.cc:2258] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:07.987618 16142 tablet_service.cc:1467] Processing CreateTablet for tablet b683e1202a064d3ab55e97150974e73d (DEFAULT_TABLE table=test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.987736 16142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b683e1202a064d3ab55e97150974e73d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.987936 16842 ts_tablet_manager.cc:1905] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:07.989240 16624 tablet_service.cc:1467] Processing CreateTablet for tablet b683e1202a064d3ab55e97150974e73d (DEFAULT_TABLE table=test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.989260 16841 ts_tablet_manager.cc:1905] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:07.989349 16624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b683e1202a064d3ab55e97150974e73d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.989822 16842 ts_tablet_manager.cc:1918] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:07.989889 16842 log.cc:1196] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/04a2e8d61f6745a5b45a2ac2c8ea05ce
I20260812 14:47:07.990185 16842 ts_tablet_manager.cc:1939] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 4ad0c22644984ead8d621351b9445f23: Deleting consensus metadata
I20260812 14:47:07.990674 16839 ts_tablet_manager.cc:1918] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:07.990747 16839 log.cc:1196] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/04a2e8d61f6745a5b45a2ac2c8ea05ce
I20260812 14:47:07.990842 16802 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Bootstrap starting.
I20260812 14:47:07.990981 16841 ts_tablet_manager.cc:1918] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:07.991021 16839 ts_tablet_manager.cc:1939] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P 956b9e2b87ea4a09815eb6e250701049: Deleting consensus metadata
I20260812 14:47:07.991046 16841 log.cc:1196] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/04a2e8d61f6745a5b45a2ac2c8ea05ce
I20260812 14:47:07.991283 16802 tablet_bootstrap.cc:654] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.991319 16841 ts_tablet_manager.cc:1939] T 04a2e8d61f6745a5b45a2ac2c8ea05ce P d12b5321e86f49f2ae7c027dc242d07a: Deleting consensus metadata
I20260812 14:47:07.991716 16801 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Bootstrap starting.
I20260812 14:47:07.992105 16801 tablet_bootstrap.cc:654] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.992482 15870 catalog_manager.cc:4841] TS 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643): tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (table test-table [id=cbf5c5f3e27c464590c547dbfb001337]) successfully deleted
I20260812 14:47:07.992623 15867 catalog_manager.cc:4841] TS 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301): tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (table test-table [id=cbf5c5f3e27c464590c547dbfb001337]) successfully deleted
I20260812 14:47:07.992669 15870 catalog_manager.cc:4841] TS d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131:36551): tablet 04a2e8d61f6745a5b45a2ac2c8ea05ce (table test-table [id=cbf5c5f3e27c464590c547dbfb001337]) successfully deleted
I20260812 14:47:07.992906 16386 tablet_service.cc:1467] Processing CreateTablet for tablet b683e1202a064d3ab55e97150974e73d (DEFAULT_TABLE table=test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:07.993031 16386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b683e1202a064d3ab55e97150974e73d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:07.993624 16801 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: No bootstrap required, opened a new log
I20260812 14:47:07.993680 16801 ts_tablet_manager.cc:1397] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:07.993834 16801 raft_consensus.cc:348] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.993913 16801 raft_consensus.cc:374] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.993937 16801 raft_consensus.cc:729] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12b5321e86f49f2ae7c027dc242d07a, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.993979 16801 consensus_queue.cc:260] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.994069 16801 ts_tablet_manager.cc:1428] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:07.994211 16807 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Bootstrap starting.
I20260812 14:47:07.994642 16802 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: No bootstrap required, opened a new log
I20260812 14:47:07.994647 16807 tablet_bootstrap.cc:654] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:07.994690 16802 ts_tablet_manager.cc:1397] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:47:07.994829 16802 raft_consensus.cc:348] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.994894 16802 raft_consensus.cc:374] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.994913 16802 raft_consensus.cc:729] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956b9e2b87ea4a09815eb6e250701049, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.994956 16802 consensus_queue.cc:260] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.995251 16802 ts_tablet_manager.cc:1428] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:07.995518 16807 tablet_bootstrap.cc:492] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: No bootstrap required, opened a new log
I20260812 14:47:07.995571 16807 ts_tablet_manager.cc:1397] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:07.995877 16807 raft_consensus.cc:348] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.995963 16807 raft_consensus.cc:374] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:07.995994 16807 raft_consensus.cc:729] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad0c22644984ead8d621351b9445f23, State: Initialized, Role: FOLLOWER
I20260812 14:47:07.996045 16807 consensus_queue.cc:260] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:07.996156 16807 ts_tablet_manager.cc:1428] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:08.063578 16811 raft_consensus.cc:482] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:08.063716 16811 raft_consensus.cc:504] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.064137 16811 leader_election.cc:288] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301), d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131:36551)
I20260812 14:47:08.067394 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.064436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:57847 (local address 127.13.146.131:36551)
0812 14:47:08.064484 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.064486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.064721 (+ 235us) server_negotiation.cc:407] Connection header received
0812 14:47:08.064807 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.064809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.064823 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.064870 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:08.065095 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.065632 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.066599 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.066957 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.067048 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.067059 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.067276 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.067321 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:08.067397 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.064346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36551 (local address 127.13.146.129:57847)
0812 14:47:08.064613 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.064626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.064732 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.064923 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.064924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.064926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:08.065010 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.065012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.065711 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.065712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.066496 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.066499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.067070 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.067071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.067149 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.067153 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.067255 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.067297 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260812 14:47:08.067922 16664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b683e1202a064d3ab55e97150974e73d" candidate_uuid: "956b9e2b87ea4a09815eb6e250701049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12b5321e86f49f2ae7c027dc242d07a" is_pre_election: true
I20260812 14:47:08.068112 16664 raft_consensus.cc:2454] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956b9e2b87ea4a09815eb6e250701049 in term 0.
I20260812 14:47:08.068172 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.064414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:43301 (local address 127.13.146.129:42833)
0812 14:47:08.064660 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.064671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.064774 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.064970 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.064970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.064973 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:08.065048 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.065051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.065987 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.065988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.067097 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.067101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.067874 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.067875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.067948 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.067952 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.068021 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.068076 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3,"thread_start_us":126,"threads_started":1}
I20260812 14:47:08.068181 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.064534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:42833 (local address 127.13.146.130:43301)
0812 14:47:08.064578 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.064580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.064745 (+ 165us) server_negotiation.cc:407] Connection header received
0812 14:47:08.064857 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.064858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.064868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.064899 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:08.065154 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.065873 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.067226 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.067760 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.067826 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.067834 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.068069 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.068113 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:08.068349 16059 leader_election.cc:302] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956b9e2b87ea4a09815eb6e250701049, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:08.068499 16811 raft_consensus.cc:2781] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:08.068455 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b683e1202a064d3ab55e97150974e73d" candidate_uuid: "956b9e2b87ea4a09815eb6e250701049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23" is_pre_election: true
I20260812 14:47:08.068526 16811 raft_consensus.cc:482] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:08.068544 16811 raft_consensus.cc:3037] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.068545 16419 raft_consensus.cc:2454] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 956b9e2b87ea4a09815eb6e250701049 in term 0.
I20260812 14:47:08.069361 16811 raft_consensus.cc:504] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.069518 16811 leader_election.cc:288] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [CANDIDATE]: Term 1 election: Requested vote from peers 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301), d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131:36551)
I20260812 14:47:08.069710 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b683e1202a064d3ab55e97150974e73d" candidate_uuid: "956b9e2b87ea4a09815eb6e250701049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23"
I20260812 14:47:08.069816 16419 raft_consensus.cc:3037] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.069746 16664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b683e1202a064d3ab55e97150974e73d" candidate_uuid: "956b9e2b87ea4a09815eb6e250701049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12b5321e86f49f2ae7c027dc242d07a"
I20260812 14:47:08.069864 16664 raft_consensus.cc:3037] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.070655 16419 raft_consensus.cc:2454] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956b9e2b87ea4a09815eb6e250701049 in term 1.
I20260812 14:47:08.070639 16664 raft_consensus.cc:2454] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 956b9e2b87ea4a09815eb6e250701049 in term 1.
I20260812 14:47:08.070861 16059 leader_election.cc:302] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956b9e2b87ea4a09815eb6e250701049, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:08.070962 16811 raft_consensus.cc:2781] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:08.071177 16811 raft_consensus.cc:686] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 1 LEADER]: Becoming Leader. State: Replica: 956b9e2b87ea4a09815eb6e250701049, State: Running, Role: LEADER
I20260812 14:47:08.071277 16811 consensus_queue.cc:237] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.072132 15892 catalog_manager.cc:5495] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 reported cstate change: term changed from 0 to 1, leader changed from <none> to 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129). New cstate: current_term: 1 leader_uuid: "956b9e2b87ea4a09815eb6e250701049" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:47:08 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546026, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:08.121049 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.114003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.13.146.190:42935)
0812 14:47:08.114147 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.114150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.114224 (+ 74us) server_negotiation.cc:407] Connection header received
0812 14:47:08.114272 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.114272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.114283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.114352 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:08.114825 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.115344 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.116013 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.116210 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.116260 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.118494 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:08.118495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:08.118495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:08.118501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:08.119883 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.120244 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.120245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.120245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.120315 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.120499 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.120500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.120500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.120612 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:08.120682 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.120895 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.120976 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260812 14:47:08.121227 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.113931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:42935 (local address 127.0.0.1:48992)
0812 14:47:08.114139 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.114150 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.114225 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.114372 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.114372 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.114702 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:08.114752 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.114755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.115416 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.115416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.115945 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.115946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.116276 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.116276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.116316 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.116921 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:08.116926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:08.118415 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:08.119963 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:08.119970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:08.119972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:08.120174 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:08.120380 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:08.120381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:08.120381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:08.120442 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:08.120704 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:08.120711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:08.120764 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.120821 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.121055 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":74,"thread_start_us":126,"threads_started":1}
I20260812 14:47:08.124156 15892 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:48992:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:47:08.124432 15892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:08.125984 16386 tablet_service.cc:1467] Processing CreateTablet for tablet 89fe9d8e723444b3b53e960150d22195 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c136f6038dfc48d2891772f10b16a869]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:08.125984 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 89fe9d8e723444b3b53e960150d22195 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c136f6038dfc48d2891772f10b16a869]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:08.126157 16386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fe9d8e723444b3b53e960150d22195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:08.126157 16624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fe9d8e723444b3b53e960150d22195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:08.127701 16807 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: Bootstrap starting.
I20260812 14:47:08.127818 16142 tablet_service.cc:1467] Processing CreateTablet for tablet 89fe9d8e723444b3b53e960150d22195 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c136f6038dfc48d2891772f10b16a869]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:08.127952 16142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fe9d8e723444b3b53e960150d22195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:08.128198 16807 tablet_bootstrap.cc:654] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:08.128521 16801 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: Bootstrap starting.
I20260812 14:47:08.128857 16801 tablet_bootstrap.cc:654] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:08.128912 16807 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: No bootstrap required, opened a new log
I20260812 14:47:08.128958 16807 ts_tablet_manager.cc:1397] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:08.129132 16807 raft_consensus.cc:348] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.129202 16802 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: Bootstrap starting.
I20260812 14:47:08.129221 16807 raft_consensus.cc:374] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:08.129266 16807 raft_consensus.cc:729] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad0c22644984ead8d621351b9445f23, State: Initialized, Role: FOLLOWER
I20260812 14:47:08.129305 16807 consensus_queue.cc:260] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.129429 16807 ts_tablet_manager.cc:1428] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:08.129597 16801 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: No bootstrap required, opened a new log
I20260812 14:47:08.129632 16801 ts_tablet_manager.cc:1397] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:08.129575 16817 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.129653 16802 tablet_bootstrap.cc:654] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:08.129667 16817 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.129720 16801 raft_consensus.cc:348] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.129768 16801 raft_consensus.cc:374] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:08.129781 16801 raft_consensus.cc:729] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12b5321e86f49f2ae7c027dc242d07a, State: Initialized, Role: FOLLOWER
I20260812 14:47:08.129841 16801 consensus_queue.cc:260] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.129944 16801 ts_tablet_manager.cc:1428] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:08.129976 16816 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.130072 16816 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.130872 16802 tablet_bootstrap.cc:492] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: No bootstrap required, opened a new log
I20260812 14:47:08.130915 16802 ts_tablet_manager.cc:1397] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:08.131071 16802 raft_consensus.cc:348] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.131150 16802 raft_consensus.cc:374] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:08.131165 16802 raft_consensus.cc:729] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956b9e2b87ea4a09815eb6e250701049, State: Initialized, Role: FOLLOWER
I20260812 14:47:08.131218 16802 consensus_queue.cc:260] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.131318 16802 ts_tablet_manager.cc:1428] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:08.131356 16811 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.131431 16811 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.407073 16816 raft_consensus.cc:482] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:08.407203 16816 raft_consensus.cc:504] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.407374 16816 leader_election.cc:288] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643), 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301)
I20260812 14:47:08.407648 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fe9d8e723444b3b53e960150d22195" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23" is_pre_election: true
I20260812 14:47:08.407752 16419 raft_consensus.cc:2454] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 0.
I20260812 14:47:08.407686 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fe9d8e723444b3b53e960150d22195" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956b9e2b87ea4a09815eb6e250701049" is_pre_election: true
I20260812 14:47:08.407801 16180 raft_consensus.cc:2454] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 0.
I20260812 14:47:08.407986 16539 leader_election.cc:302] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956b9e2b87ea4a09815eb6e250701049, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:08.408100 16816 raft_consensus.cc:2781] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:08.408162 16816 raft_consensus.cc:482] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:08.408190 16816 raft_consensus.cc:3037] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.408876 16816 raft_consensus.cc:504] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.409029 16816 leader_election.cc:288] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 election: Requested vote from peers 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643), 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301)
I20260812 14:47:08.409191 16419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fe9d8e723444b3b53e960150d22195" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad0c22644984ead8d621351b9445f23"
I20260812 14:47:08.409302 16419 raft_consensus.cc:3037] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.409282 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fe9d8e723444b3b53e960150d22195" candidate_uuid: "d12b5321e86f49f2ae7c027dc242d07a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956b9e2b87ea4a09815eb6e250701049"
I20260812 14:47:08.409353 16180 raft_consensus.cc:3037] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:08.410086 16419 raft_consensus.cc:2454] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 1.
I20260812 14:47:08.410101 16180 raft_consensus.cc:2454] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d12b5321e86f49f2ae7c027dc242d07a in term 1.
I20260812 14:47:08.410262 16539 leader_election.cc:302] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 956b9e2b87ea4a09815eb6e250701049, d12b5321e86f49f2ae7c027dc242d07a; no voters:
I20260812 14:47:08.410370 16816 raft_consensus.cc:2781] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:08.410444 16816 raft_consensus.cc:686] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [term 1 LEADER]: Becoming Leader. State: Replica: d12b5321e86f49f2ae7c027dc242d07a, State: Running, Role: LEADER
I20260812 14:47:08.410496 16816 consensus_queue.cc:237] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } }
I20260812 14:47:08.410772 16820 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: TxnStatusTablet state changed. Reason: New leader d12b5321e86f49f2ae7c027dc242d07a. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.410872 16820 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:08.411046 16893 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:47:08.411183 16893 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:47:08.411091 15892 catalog_manager.cc:5495] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a reported cstate change: term changed from 0 to 1, leader changed from <none> to d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131). New cstate: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:08.486933 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.484023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.13.146.131:36551)
0812 14:47:08.484072 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.484073 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.484128 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:47:08.484228 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.484229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.484237 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.484287 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:08.484422 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.484946 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.486187 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.486424 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.486479 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.486714 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.486744 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.486782 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.486849 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.486868 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:47:08.486991 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.483964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36551 (local address 127.0.0.1:43102)
0812 14:47:08.484077 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.484089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.484151 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.484298 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.484298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.484300 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:08.484361 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.484364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.485048 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.485050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.486056 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.486060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.486505 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.486506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.486573 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.486597 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.486787 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.486794 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.486846 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.486906 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 14:47:08.488204 16180 raft_consensus.cc:1261] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Refusing update from remote peer d12b5321e86f49f2ae7c027dc242d07a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:08.488204 16419 raft_consensus.cc:1261] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Refusing update from remote peer d12b5321e86f49f2ae7c027dc242d07a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:08.488436 16816 consensus_queue.cc:1035] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Connected to new peer: Peer: permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:08.488538 16820 consensus_queue.cc:1035] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:08.489046 16811 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: TxnStatusTablet state changed. Reason: New leader d12b5321e86f49f2ae7c027dc242d07a. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.489140 16811 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.489074 16817 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: TxnStatusTablet state changed. Reason: New leader d12b5321e86f49f2ae7c027dc242d07a. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.489176 16817 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.490082 16820 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.490187 16820 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:08.490396 16820 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.490466 16820 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P d12b5321e86f49f2ae7c027dc242d07a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:08.490720 16811 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.490808 16811 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 956b9e2b87ea4a09815eb6e250701049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.491866 16817 tablet_replica.cc:439] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d12b5321e86f49f2ae7c027dc242d07a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956b9e2b87ea4a09815eb6e250701049" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 42643 } } peers { permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 } } peers { permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 } } }
I20260812 14:47:08.491964 16817 tablet_replica.cc:442] T 89fe9d8e723444b3b53e960150d22195 P 4ad0c22644984ead8d621351b9445f23: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:08.495764 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.492705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.13.146.129:42643)
0812 14:47:08.492894 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.492896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.492902 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:47:08.492933 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.492933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.492951 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.493010 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:08.493257 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.493744 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.494851 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.495043 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.495098 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.495303 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.495456 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.495556 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.495649 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.495676 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":57,"threads_started":1}
I20260812 14:47:08.501080 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.498578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:40223 (local address 127.13.146.131:36551)
0812 14:47:08.498630 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.498634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.498777 (+ 143us) server_negotiation.cc:407] Connection header received
0812 14:47:08.498847 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.498849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.498862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.498917 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:08.499075 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.499561 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.500331 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.500526 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.500603 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.500799 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.500836 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.500901 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.500975 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.500997 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:47:08.501147 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.498508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36551 (local address 127.13.146.129:40223)
0812 14:47:08.498707 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.498717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.498793 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.498921 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.498922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.498923 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:08.499001 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.499003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.499653 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.499654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.500240 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.500242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.500603 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.500604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.500682 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.500700 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.500902 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.500905 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.500969 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.501051 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260812 14:47:08.503696 16419 raft_consensus.cc:1261] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Refusing update from remote peer 956b9e2b87ea4a09815eb6e250701049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:08.503794 16664 raft_consensus.cc:1261] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Refusing update from remote peer 956b9e2b87ea4a09815eb6e250701049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:08.504000 16863 consensus_queue.cc:1035] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ad0c22644984ead8d621351b9445f23" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 43301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:08.504314 16863 consensus_queue.cc:1035] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d12b5321e86f49f2ae7c027dc242d07a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:08.514194 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.511236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:53955 (local address 127.13.146.129:42643)
0812 14:47:08.511285 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.511287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.511398 (+ 111us) server_negotiation.cc:407] Connection header received
0812 14:47:08.511428 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.511429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.511440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.511510 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:08.511617 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.512352 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.513427 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.513716 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.513797 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.513942 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.513981 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.514048 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.514087 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.514111 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:08.514225 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.511197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:42643 (local address 127.13.146.131:53955)
0812 14:47:08.511306 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:08.511317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:08.511381 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:08.511511 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:08.511512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:08.511514 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:08.511576 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.511577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.512433 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.512434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.513345 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:08.513348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.513794 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.513795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:08.513844 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.513859 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:08.514043 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:08.514045 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:08.514080 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 14:47:08.514145 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":50,"threads_started":1}
I20260812 14:47:08.523902 15890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:08.524048 15890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:08.524744 15890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d77bad5d05e479b80054fb30abe0743: Sending DeleteTablet for 3 replicas of tablet b683e1202a064d3ab55e97150974e73d
I20260812 14:47:08.525029 16624 tablet_service.cc:1514] Processing DeleteTablet for tablet b683e1202a064d3ab55e97150974e73d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:08 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:43092
I20260812 14:47:08.525063 16386 tablet_service.cc:1514] Processing DeleteTablet for tablet b683e1202a064d3ab55e97150974e73d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:08 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56322
I20260812 14:47:08.525348 16937 tablet_replica.cc:330] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: stopping tablet replica
I20260812 14:47:08.525339 16142 tablet_service.cc:1514] Processing DeleteTablet for tablet b683e1202a064d3ab55e97150974e73d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:08 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57224
I20260812 14:47:08.525409 16937 raft_consensus.cc:2229] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:08.525449 16937 raft_consensus.cc:2258] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:08.525727 16937 ts_tablet_manager.cc:1905] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:08.526346 16940 tablet_replica.cc:330] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: stopping tablet replica
I20260812 14:47:08.526499 16941 tablet_replica.cc:330] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: stopping tablet replica
I20260812 14:47:08.526868 16941 raft_consensus.cc:2229] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:08.526989 16940 raft_consensus.cc:2229] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:08.527318 16941 raft_consensus.cc:2258] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:08.527443 16940 raft_consensus.cc:2258] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:08.527696 16937 ts_tablet_manager.cc:1918] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:08.527772 16937 log.cc:1196] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/b683e1202a064d3ab55e97150974e73d
I20260812 14:47:08.528013 16937 ts_tablet_manager.cc:1939] T b683e1202a064d3ab55e97150974e73d P d12b5321e86f49f2ae7c027dc242d07a: Deleting consensus metadata
I20260812 14:47:08.528195 16940 ts_tablet_manager.cc:1905] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:08.528348 16941 ts_tablet_manager.cc:1905] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:08.528352 15870 catalog_manager.cc:4841] TS d12b5321e86f49f2ae7c027dc242d07a (127.13.146.131:36551): tablet b683e1202a064d3ab55e97150974e73d (table test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]) successfully deleted
I20260812 14:47:08.530167 16940 ts_tablet_manager.cc:1918] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:08.530290 16940 log.cc:1196] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/b683e1202a064d3ab55e97150974e73d
I20260812 14:47:08.530614 16940 ts_tablet_manager.cc:1939] T b683e1202a064d3ab55e97150974e73d P 4ad0c22644984ead8d621351b9445f23: Deleting consensus metadata
I20260812 14:47:08.530959 15867 catalog_manager.cc:4841] TS 4ad0c22644984ead8d621351b9445f23 (127.13.146.130:43301): tablet b683e1202a064d3ab55e97150974e73d (table test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]) successfully deleted
I20260812 14:47:08.531226 16941 ts_tablet_manager.cc:1918] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:08.531293 16941 log.cc:1196] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/b683e1202a064d3ab55e97150974e73d
I20260812 14:47:08.531544 16941 ts_tablet_manager.cc:1939] T b683e1202a064d3ab55e97150974e73d P 956b9e2b87ea4a09815eb6e250701049: Deleting consensus metadata
I20260812 14:47:08.531903 15870 catalog_manager.cc:4841] TS 956b9e2b87ea4a09815eb6e250701049 (127.13.146.129:42643): tablet b683e1202a064d3ab55e97150974e73d (table test-table [id=e7321ddbdc3d42fca62c3e0778a0ff62]) successfully deleted
Aug 12 14:47:08 dist-test-slave-8hhm krb5kdc[15817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546027, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:08.548566 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.526789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.13.146.129:42643)
0812 14:47:08.526829 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.526831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.531929 (+ 5098us) server_negotiation.cc:407] Connection header received
0812 14:47:08.532051 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.532052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.532067 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.532129 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:08.532842 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.533613 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.534709 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.535008 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.535091 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.545389 (+ 10298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:08.545403 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:08.545404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:08.545414 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:08.547236 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.547658 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.547661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.547661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.547734 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.547942 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.547943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.547943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.548124 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:08.548320 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.548382 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.548458 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:47:08.548830 16240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57236
I20260812 14:47:08.556577 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:08.549944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.13.146.190:42935)
0812 14:47:08.550004 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:08.550007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:08.550017 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:08.550085 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:08.550086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:08.550100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:08.550173 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:08.550645 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.551458 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.552158 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:08.552485 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:08.552558 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:08.553836 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:08.553837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:08.553837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:08.553845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:08.555384 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.555710 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.555711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.555712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.555778 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:08.556029 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:08.556030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:08.556031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:08.556179 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:08.556285 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:08.556383 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:47:08.556477 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 14:47:08.556797 15890 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996
I20260812 14:47:08.557600 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16021
I20260812 14:47:08.582123 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16261
I20260812 14:47:08.598321 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16505
I20260812 14:47:08.613497 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15836
2026-08-12T14:47:08Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3265 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:08 dist-test-slave-8hhm krb5kdc[16949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:08 dist-test-slave-8hhm krb5kdc[16949](info): set up 2 sockets
Aug 12 14:47:08 dist-test-slave-8hhm krb5kdc[16949](info): commencing operation
krb5kdc: starting...
W20260812 14:47:09.945364 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:10.070523 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37545
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:41925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:37545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:10.154922 16974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:10.155153 16974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:10.155176 16974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:10.156351 16974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:10.156396 16974 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:47:10.156409 16974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:10.156422 16974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:10.156435 16974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:10.156446 16974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:10.158077 16974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:41925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:37545
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:10.158357 16974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:10.158599 16974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:10.163790 16983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.163825 16986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.163867 16974 server_base.cc:1034] running on GCE node
W20260812 14:47:10.163803 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.164382 16974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:10.164675 16974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:10.165897 16974 hybrid_clock.cc:648] HybridClock initialized: now 1786546030165887 us; error 38 us; skew 500 ppm
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:10.172842 16974 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:10.173350 16974 webserver.cc:460] Webserver started at http://127.13.146.190:34665/ using document root <none> and password file <none>
I20260812 14:47:10.173579 16974 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:10.173621 16974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:10.173717 16974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:10.174768 16974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "e710dba17be4458dac126c662b869d60"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "c2a6be1557a2fe783d213356ef15693e"
server_key_iv: "811632ee098b4fa9b9bfcce2316e4d50"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.175113 16974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "e710dba17be4458dac126c662b869d60"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "c2a6be1557a2fe783d213356ef15693e"
server_key_iv: "811632ee098b4fa9b9bfcce2316e4d50"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.176491 16974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:10.177250 16997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.177467 16974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:10.177582 16974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "e710dba17be4458dac126c662b869d60"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "c2a6be1557a2fe783d213356ef15693e"
server_key_iv: "811632ee098b4fa9b9bfcce2316e4d50"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.177672 16974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:10.207026 16974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:10.209640 16974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:10.209822 16974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:10.214808 16974 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:37545
I20260812 14:47:10.214859 17121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:37545 every 8 connection(s)
I20260812 14:47:10.215219 16974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:10.215924 17122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:10.216329 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16974
I20260812 14:47:10.216418 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:10.216624 13898 external_mini_cluster.cc:1428] Setting key e88c943f7d88d452170b197cc53f4314
I20260812 14:47:10.219151 17122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Bootstrap starting.
I20260812 14:47:10.219880 17122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:10.220218 17122 log.cc:824] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:10.221001 17122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: No bootstrap required, opened a new log
I20260812 14:47:10.223197 17122 raft_consensus.cc:348] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } }
I20260812 14:47:10.223388 17122 raft_consensus.cc:374] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:10.223444 17122 raft_consensus.cc:729] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e710dba17be4458dac126c662b869d60, State: Initialized, Role: FOLLOWER
I20260812 14:47:10.223626 17122 consensus_queue.cc:260] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } }
I20260812 14:47:10.223737 17122 raft_consensus.cc:388] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:10.223788 17122 raft_consensus.cc:482] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:10.223845 17122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:10.224663 17122 raft_consensus.cc:504] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } }
I20260812 14:47:10.224805 17122 leader_election.cc:302] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e710dba17be4458dac126c662b869d60; no voters:
I20260812 14:47:10.225001 17122 leader_election.cc:288] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:10.225131 17129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:10.225343 17122 sys_catalog.cc:564] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:10.225363 17129 raft_consensus.cc:686] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [term 1 LEADER]: Becoming Leader. State: Replica: e710dba17be4458dac126c662b869d60, State: Running, Role: LEADER
I20260812 14:47:10.225555 17129 consensus_queue.cc:237] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } }
I20260812 14:47:10.225980 17132 sys_catalog.cc:455] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e710dba17be4458dac126c662b869d60. Latest consensus state: current_term: 1 leader_uuid: "e710dba17be4458dac126c662b869d60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } } }
I20260812 14:47:10.226069 17132 sys_catalog.cc:458] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:10.226290 17144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:10.226853 17144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:10.227757 17130 sys_catalog.cc:455] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e710dba17be4458dac126c662b869d60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e710dba17be4458dac126c662b869d60" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37545 } } }
I20260812 14:47:10.227844 17130 sys_catalog.cc:458] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:10.229033 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.217303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.13.146.190:37545)
0812 14:47:10.217512 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.217523 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.217561 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:47:10.218103 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.218113 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.218176 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.218335 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.219093 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.220493 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.221357 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.221776 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.221898 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.224566 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.224570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.224571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.224582 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.227233 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.227709 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.227712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.227713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.227830 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.228056 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.228058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.228058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.228388 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.228542 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.228714 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.228862 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260812 14:47:10.233691 17144 catalog_manager.cc:1347] Generated new cluster ID: e9c6a33738d84b8fb2100652d7b7ac5e
I20260812 14:47:10.233742 17144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:10.252562 17144 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:10.253165 17144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:10.260401 17144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Generated new TSK 0
I20260812 14:47:10.260629 17144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:10.301031 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37545
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:41925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:10.407442 17166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:10.407765 17166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:10.407799 17166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:10.409054 17166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:10.409101 17166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:10.409124 17166 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:47:10.409193 17166 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:10.409252 17166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:10.410990 17166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:41925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:10.411284 17166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:10.411550 17166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:10.417691 17180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.417676 17182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.417716 17166 server_base.cc:1034] running on GCE node
W20260812 14:47:10.417676 17179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.418226 17166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:10.418498 17166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:10.419675 17166 hybrid_clock.cc:648] HybridClock initialized: now 1786546030419648 us; error 43 us; skew 500 ppm
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:10.422605 17166 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:10.423054 17166 webserver.cc:460] Webserver started at http://127.13.146.129:42377/ using document root <none> and password file <none>
I20260812 14:47:10.423276 17166 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:10.423321 17166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:10.423427 17166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:10.424548 17166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "2a6ec476acb6478987ca782cbc968b3c"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "44e9b50802b281708aa84254b9bd3557"
server_key_iv: "7e30a06df45c5fda66d9d1dff3b3c198"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.424952 17166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "2a6ec476acb6478987ca782cbc968b3c"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "44e9b50802b281708aa84254b9bd3557"
server_key_iv: "7e30a06df45c5fda66d9d1dff3b3c198"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.426708 17166 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 14:47:10.427597 17196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.427812 17166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:10.427958 17166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "2a6ec476acb6478987ca782cbc968b3c"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "44e9b50802b281708aa84254b9bd3557"
server_key_iv: "7e30a06df45c5fda66d9d1dff3b3c198"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.428081 17166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:10.447707 17166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:10.449980 17166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:10.450201 17166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:10.450796 17166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:10.450868 17166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.450922 17166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:10.450984 17166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:10.459417 17166 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:40789
I20260812 14:47:10.459769 17407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:40789 every 8 connection(s)
I20260812 14:47:10.459837 17166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:10.462275 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.451381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:59499 (local address 127.13.146.190:37545)
0812 14:47:10.451474 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.451477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.451796 (+ 319us) server_negotiation.cc:407] Connection header received
0812 14:47:10.452681 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.452682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.452702 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.452748 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.453859 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.454384 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.456215 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.456441 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.456520 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.459078 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.459079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.459079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.459104 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.460921 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.461383 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.461384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.461384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.461444 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.461730 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.461730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.461731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.461845 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.461899 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.462131 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.462205 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:10.462396 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.451341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.129:59499)
0812 14:47:10.451610 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.451676 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.452536 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.452854 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.452859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.453369 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.453737 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.453747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.454468 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.454469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.456116 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.456120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.456562 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.456563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.456713 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.457279 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.457288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.458978 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.460997 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.461003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.461005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.461296 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.461515 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.461515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.461516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.461659 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.461902 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.461907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.462019 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.462097 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.462221 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":127,"threads_started":1}
I20260812 14:47:10.462988 17411 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37545
I20260812 14:47:10.463071 17411 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:10.463256 17411 heartbeater.cc:511] Master 127.13.146.190:37545 requested a full tablet report, sending...
I20260812 14:47:10.463816 17031 ts_manager.cc:194] Registered new tserver with Master: 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789)
I20260812 14:47:10.464646 17031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:59499
I20260812 14:47:10.468585 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17166
I20260812 14:47:10.468721 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:10.468914 13898 external_mini_cluster.cc:1428] Setting key 6ec39f222898ab5aa082687e93971f7d
I20260812 14:47:10.470000 17425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.464123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.129:60639)
0812 14:47:10.464262 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.464273 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.464356 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.464495 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.464496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.464688 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.464739 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.464741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.465580 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.465580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.466047 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.466048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.466415 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.466416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.466503 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.467011 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.467017 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.467332 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.468787 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.468788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.468789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.468967 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.469263 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.469264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.469265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.469336 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.469645 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.469646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.469701 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.469783 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.469898 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 14:47:10.470023 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.464174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:60639 (local address 127.13.146.190:37545)
0812 14:47:10.464210 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.464212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.464340 (+ 128us) server_negotiation.cc:407] Connection header received
0812 14:47:10.464412 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.464413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.464433 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.464490 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.464828 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.465508 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.466124 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.466317 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.466377 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.467428 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.467430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.467430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.467436 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.468665 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.469071 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.469073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.469073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.469145 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.469413 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.469414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.469414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.469558 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.469639 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.469814 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.469897 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:10.495565 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37545
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:41925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:10.579276 17430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:10.579535 17430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:10.579577 17430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:10.580853 17430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:10.580917 17430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:10.580953 17430 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:47:10.580999 17430 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:10.581032 17430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:10.582584 17430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:41925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:10.582851 17430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:10.583104 17430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:10.588505 17445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.588523 17457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.588521 17438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.588608 17430 server_base.cc:1034] running on GCE node
I20260812 14:47:10.588908 17430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:10.589159 17430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:10.590305 17430 hybrid_clock.cc:648] HybridClock initialized: now 1786546030590289 us; error 31 us; skew 500 ppm
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:10.592958 17430 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:10.593500 17430 webserver.cc:460] Webserver started at http://127.13.146.130:43163/ using document root <none> and password file <none>
I20260812 14:47:10.593732 17430 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:10.593802 17430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:10.593955 17430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:10.595021 17430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "59e34a2e80f1446ebf8651fd49167249"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "4f378a8e040d52b603b7dc3cf44dd4c0"
server_key_iv: "240a9154976ca6b737a4f5a7b68c0085"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.595404 17430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "59e34a2e80f1446ebf8651fd49167249"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "4f378a8e040d52b603b7dc3cf44dd4c0"
server_key_iv: "240a9154976ca6b737a4f5a7b68c0085"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.596758 17430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:10.597527 17468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.597688 17430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:10.597796 17430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "59e34a2e80f1446ebf8651fd49167249"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "4f378a8e040d52b603b7dc3cf44dd4c0"
server_key_iv: "240a9154976ca6b737a4f5a7b68c0085"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.597869 17430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:10.608942 17430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:10.611344 17430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:10.611527 17430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:10.612128 17430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:10.612190 17430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.612232 17430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:10.612255 17430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.619930 17430 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:42293
I20260812 14:47:10.620019 17677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:42293 every 8 connection(s)
I20260812 14:47:10.620321 17430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:10.621071 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17430
I20260812 14:47:10.621201 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:10.621474 13898 external_mini_cluster.cc:1428] Setting key 651da0a42e27789c299df616de67feea
I20260812 14:47:10.624372 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.612546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:49677 (local address 127.13.146.190:37545)
0812 14:47:10.612633 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.612636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.612897 (+ 261us) server_negotiation.cc:407] Connection header received
0812 14:47:10.613717 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.613718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.613750 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.613829 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.614760 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.615312 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.616978 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.617199 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.617341 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.621005 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.621006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.621006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.621014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.622630 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.623246 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.623247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.623247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.623321 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.623619 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.623619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.623619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.623731 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.623786 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.624160 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.624230 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 14:47:10.624642 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.612532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.130:49677)
0812 14:47:10.612733 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.612787 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.613590 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.613925 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.613929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.614326 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.614653 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.614660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.615414 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.615415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.616876 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.616879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.617354 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.617355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.617525 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.618772 (+ 1247us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.618782 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.620877 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.622762 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.622770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.622772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.623157 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.623373 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.623373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.623374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.623546 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.623806 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.623814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.624004 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.624136 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.624349 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
I20260812 14:47:10.625171 17678 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37545
I20260812 14:47:10.625299 17678 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:10.625532 17678 heartbeater.cc:511] Master 127.13.146.190:37545 requested a full tablet report, sending...
I20260812 14:47:10.625950 17032 ts_manager.cc:194] Registered new tserver with Master: 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293)
I20260812 14:47:10.626470 17032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:49677
I20260812 14:47:10.632855 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.626387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.130:57823)
0812 14:47:10.626540 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.626552 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.626612 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.626805 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.626806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.627060 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.627135 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.627137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.627779 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.627780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.628288 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.628290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.628661 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.628662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.628736 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.629293 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.629299 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.629680 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.631723 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.631724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.631725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.631904 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.632158 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.632158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.632159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.632217 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.632608 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.632609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.632648 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.632701 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.632778 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260812 14:47:10.632934 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.626648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57823 (local address 127.13.146.190:37545)
0812 14:47:10.626683 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.626685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.626693 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:10.626713 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.626713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.626723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.626782 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.627218 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.627677 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.628376 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.628585 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.628662 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.629786 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.629787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.629788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.629797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.631635 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.631990 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.631992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.631992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.632076 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.632288 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.632289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.632290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.632523 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.632618 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.632736 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.632837 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:10.652390 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37545
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:41925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:10.740397 17701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:10.740607 17701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:10.740631 17701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:10.741827 17701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:10.741873 17701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:10.741892 17701 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:47:10.741904 17701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:10.741921 17701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:10.743309 17701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:41925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:37545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:10.743497 17701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:10.743669 17701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:10.748993 17709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.749027 17713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:10.749006 17707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:10.749161 17701 server_base.cc:1034] running on GCE node
I20260812 14:47:10.749440 17701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:10.749648 17701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:10.750813 17701 hybrid_clock.cc:648] HybridClock initialized: now 1786546030750795 us; error 33 us; skew 500 ppm
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:10.753397 17701 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:10.753846 17701 webserver.cc:460] Webserver started at http://127.13.146.131:40281/ using document root <none> and password file <none>
I20260812 14:47:10.754034 17701 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:10.754076 17701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:10.754177 17701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:10.755201 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "f9a9352115f98982a4791fb49321003c"
server_key_iv: "5916bd54f34474688e83beb0f42342e4"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.755506 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "f9a9352115f98982a4791fb49321003c"
server_key_iv: "5916bd54f34474688e83beb0f42342e4"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.756866 17701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:10.757659 17726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.757843 17701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.757963 17701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
format_stamp: "Formatted at 2026-08-12 14:47:10 on dist-test-slave-8hhm"
server_key: "f9a9352115f98982a4791fb49321003c"
server_key_iv: "5916bd54f34474688e83beb0f42342e4"
server_key_version: "encryptionkey@0"
I20260812 14:47:10.758019 17701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:10.785058 17701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:10.787185 17701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:10.787349 17701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:10.787921 17701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:10.787963 17701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.787992 17701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:10.788007 17701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:10.795421 17701 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:34335
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:10.795490 17955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:34335 every 8 connection(s)
I20260812 14:47:10.795830 17701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:10.798791 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17701
I20260812 14:47:10.798878 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:10.798962 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.788607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:49881 (local address 127.13.146.190:37545)
0812 14:47:10.788671 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.788673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.788896 (+ 223us) server_negotiation.cc:407] Connection header received
0812 14:47:10.789550 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.789551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.789572 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.789624 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.790563 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.791453 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.793121 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.793421 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.793512 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.795971 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.795972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.795973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.795980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.797458 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.797889 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.797890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.797890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.797956 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.798227 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.798228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.798228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.798353 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.798425 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.798758 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.798834 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:47:10.799078 13898 external_mini_cluster.cc:1428] Setting key d3831f0b3fd3a3a88e53359eb90b2a16
I20260812 14:47:10.799139 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.788579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.131:49881)
0812 14:47:10.788764 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.788802 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.789427 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.789713 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.789717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.790115 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.790434 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.790442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.791570 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.791571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.793014 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.793017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.793524 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.793525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.793666 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.794297 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.794318 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.795866 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.797559 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.797566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.797568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.797798 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.798035 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.798035 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.798036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.798150 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.798428 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.798435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.798581 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.798716 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.798905 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260812 14:47:10.799671 17956 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37545
I20260812 14:47:10.799767 17956 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:10.799983 17956 heartbeater.cc:511] Master 127.13.146.190:37545 requested a full tablet report, sending...
I20260812 14:47:10.800401 17032 ts_manager.cc:194] Registered new tserver with Master: 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:10.800772 17032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:49881
I20260812 14:47:10.806139 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.800803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.13.146.131:51465)
0812 14:47:10.800939 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.800952 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.801014 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.801149 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.801150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.801400 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:10.801496 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.801499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.802130 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.802132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.802629 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.802631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.802977 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.802978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.803051 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.803536 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:10.803541 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:10.803836 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:10.805269 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.805270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.805270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.805458 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.805650 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:10.805651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:10.805651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:10.805707 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:10.805912 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:10.805913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:10.805948 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.805998 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.806073 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260812 14:47:10.806154 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.800856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51465 (local address 127.13.146.190:37545)
0812 14:47:10.800894 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.800896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.801017 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:47:10.801075 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.801075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.801086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.801144 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.801576 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.802021 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.802705 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.802881 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.802934 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.803920 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.803921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.803921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.803927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.805143 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.805529 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.805529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.805530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.805585 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.805762 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.805762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.805763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.805858 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.805908 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.806015 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.806071 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:47:10.810801 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:47:10 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:10.835632 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.828306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.13.146.190:37545)
0812 14:47:10.828373 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.828375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.828385 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:10.828458 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.828459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.828468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.828521 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:10.829109 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.829701 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.830472 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.830680 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.830734 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.833149 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:10.833151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:10.833151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:10.833158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:10.834647 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.834980 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.834981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.834982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.835039 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:10.835234 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:10.835235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:10.835235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:10.835340 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:10.835415 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.835500 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.835560 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:47:10.837313 17032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:10.838938 17032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:10.845378 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.13.146.130:42293)
0812 14:47:10.841793 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.841798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.841808 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:10.841845 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.841847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.841861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.841933 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:10.842189 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.842928 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.844328 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.844954 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845042 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845112 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.845243 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.845281 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":130,"threads_started":1}
I20260812 14:47:10.845368 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.0.0.1:34958)
0812 14:47:10.841529 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.841577 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.841643 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.841941 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.841944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.841946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:10.842078 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.842080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.843021 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.843022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.844237 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.844240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.845039 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.845127 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845141 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.845240 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.845290 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260812 14:47:10.845813 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:40789 (local address 127.0.0.1:56778)
0812 14:47:10.842041 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.842052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.842115 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.842377 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.842378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.842380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:10.842423 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.842425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.843618 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.843623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.844621 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.844624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.845601 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.845658 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845662 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.845706 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.845744 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":83,"threads_started":1}
I20260812 14:47:10.846026 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.13.146.131:34335)
0812 14:47:10.841527 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.841533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.841706 (+ 173us) server_negotiation.cc:407] Connection header received
0812 14:47:10.841795 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.841797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.841814 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.841886 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:10.842150 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.843314 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.844512 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.845200 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845332 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845388 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.845886 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.845928 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260812 14:47:10.846146 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.13.146.129:40789)
0812 14:47:10.841886 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:10.841890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:10.842176 (+ 286us) server_negotiation.cc:407] Connection header received
0812 14:47:10.842225 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:10.842227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:10.842242 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:10.842311 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:10.842501 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.843530 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.844710 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.845450 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845522 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845572 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:10.845983 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 14:47:10.846025 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260812 14:47:10.846123 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 9adac297c23c4c3687a90c168d221b07 (DEFAULT_TABLE table=test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:10.846318 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:10.841218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:34335 (local address 127.0.0.1:58472)
0812 14:47:10.841617 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:10.841626 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:10.841712 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:10.841959 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:10.841960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:10.841962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:10.842066 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.842068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.843420 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.843421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.844415 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:10.844417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:10.845415 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:10.845743 (+ 328us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:10.845797 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:10.845800 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:10.846019 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 14:47:10.846058 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260812 14:47:10.846544 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adac297c23c4c3687a90c168d221b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:10.846755 17290 tablet_service.cc:1467] Processing CreateTablet for tablet 9adac297c23c4c3687a90c168d221b07 (DEFAULT_TABLE table=test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:10.846909 17822 tablet_service.cc:1467] Processing CreateTablet for tablet 9adac297c23c4c3687a90c168d221b07 (DEFAULT_TABLE table=test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:10.847160 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adac297c23c4c3687a90c168d221b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:10.847259 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9adac297c23c4c3687a90c168d221b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:10.849671 17997 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Bootstrap starting.
I20260812 14:47:10.850294 17997 tablet_bootstrap.cc:654] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:10.850533 18001 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Bootstrap starting.
I20260812 14:47:10.850663 17997 log.cc:824] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:10.851033 18004 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Bootstrap starting.
I20260812 14:47:10.851475 17997 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: No bootstrap required, opened a new log
I20260812 14:47:10.851549 17997 ts_tablet_manager.cc:1397] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:10.851575 18001 tablet_bootstrap.cc:654] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:10.851727 18004 tablet_bootstrap.cc:654] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:10.851958 18001 log.cc:824] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:10.852080 18004 log.cc:824] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:10.852845 18001 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: No bootstrap required, opened a new log
I20260812 14:47:10.852917 18001 ts_tablet_manager.cc:1397] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:10.853004 18004 tablet_bootstrap.cc:492] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: No bootstrap required, opened a new log
I20260812 14:47:10.853075 18004 ts_tablet_manager.cc:1397] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:10.853787 17997 raft_consensus.cc:348] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.853967 17997 raft_consensus.cc:374] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:10.854007 17997 raft_consensus.cc:729] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Initialized, Role: FOLLOWER
I20260812 14:47:10.854149 17997 consensus_queue.cc:260] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.854423 17997 ts_tablet_manager.cc:1428] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:10.854548 17678 heartbeater.cc:503] Master 127.13.146.190:37545 was elected leader, sending a full tablet report...
I20260812 14:47:10.855005 18001 raft_consensus.cc:348] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.855134 18001 raft_consensus.cc:374] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:10.855155 18001 raft_consensus.cc:729] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Initialized, Role: FOLLOWER
I20260812 14:47:10.854837 18004 raft_consensus.cc:348] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.855276 18001 consensus_queue.cc:260] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.855368 18004 raft_consensus.cc:374] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:10.855422 18004 raft_consensus.cc:729] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Initialized, Role: FOLLOWER
I20260812 14:47:10.855542 18001 ts_tablet_manager.cc:1428] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:10.855541 18004 consensus_queue.cc:260] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:10.855695 17956 heartbeater.cc:503] Master 127.13.146.190:37545 was elected leader, sending a full tablet report...
I20260812 14:47:10.855933 18004 ts_tablet_manager.cc:1428] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:47:10.855993 17411 heartbeater.cc:503] Master 127.13.146.190:37545 was elected leader, sending a full tablet report...
W20260812 14:47:10.870891 17679 tablet.cc:2369] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:47:10.960858 17413 tablet.cc:2369] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:11.017664 18014 raft_consensus.cc:482] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:11.017809 18014 raft_consensus.cc:504] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:11.018232 18014 leader_election.cc:288] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:11.021330 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.018630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:39099 (local address 127.13.146.131:34335)
0812 14:47:11.018686 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.018688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.018701 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:11.018783 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.018784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.018792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.018868 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.019112 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.019554 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.020553 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.020930 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.020987 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.020994 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.021191 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.021260 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:47:11.021369 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.018555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:37991 (local address 127.13.146.130:42293)
0812 14:47:11.018614 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.018617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.018718 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:47:11.018762 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.018763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.018775 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.018833 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.019026 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.019533 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.020587 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.020941 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.021033 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.021041 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.021256 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.021297 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:47:11.021373 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.018438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.13.146.129:37991)
0812 14:47:11.018614 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.018627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.018706 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.018852 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.018853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.018856 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.018937 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.018944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.019621 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.019623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.020460 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.020463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.021049 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.021050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.021140 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.021163 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.021250 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.021289 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":63,"thread_start_us":101,"threads_started":1}
I20260812 14:47:11.021343 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.018473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:34335 (local address 127.13.146.129:39099)
0812 14:47:11.018632 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.018644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.018711 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.018891 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.018893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.018896 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.018963 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.018964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.019668 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.019669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.020456 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.020458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.021029 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.021030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.021111 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.021115 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.021158 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.021192 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":14,"thread_start_us":76,"threads_started":1}
I20260812 14:47:11.021785 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adac297c23c4c3687a90c168d221b07" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249" is_pre_election: true
I20260812 14:47:11.021970 17598 raft_consensus.cc:2454] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 0.
I20260812 14:47:11.021996 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adac297c23c4c3687a90c168d221b07" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" is_pre_election: true
I20260812 14:47:11.022158 17858 raft_consensus.cc:2454] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 0.
I20260812 14:47:11.022219 17203 leader_election.cc:302] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 59e34a2e80f1446ebf8651fd49167249; no voters:
I20260812 14:47:11.022352 18014 raft_consensus.cc:2781] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:11.022439 18014 raft_consensus.cc:482] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:11.022477 18014 raft_consensus.cc:3037] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.023238 18014 raft_consensus.cc:504] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:11.023388 18014 leader_election.cc:288] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 election: Requested vote from peers 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:11.023710 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adac297c23c4c3687a90c168d221b07" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249"
I20260812 14:47:11.023710 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adac297c23c4c3687a90c168d221b07" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
I20260812 14:47:11.023794 17598 raft_consensus.cc:3037] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.023794 17858 raft_consensus.cc:3037] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.024681 17598 raft_consensus.cc:2454] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 1.
I20260812 14:47:11.024681 17858 raft_consensus.cc:2454] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 1.
I20260812 14:47:11.024883 17198 leader_election.cc:302] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 48725a06e6fd4e6abf7aa4cf1680b0cd; no voters:
I20260812 14:47:11.025020 18014 raft_consensus.cc:2781] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:11.025254 18014 raft_consensus.cc:686] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 1 LEADER]: Becoming Leader. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Running, Role: LEADER
I20260812 14:47:11.025359 18014 consensus_queue.cc:237] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } }
I20260812 14:47:11.026271 17031 catalog_manager.cc:5495] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129). New cstate: current_term: 1 leader_uuid: "2a6ec476acb6478987ca782cbc968b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } health_report { overall_health: HEALTHY } } }
W20260812 14:47:11.046559 17957 tablet.cc:2369] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 14:47:11 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546030, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:11.097623 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.090081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.13.146.190:37545)
0812 14:47:11.090186 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.090189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.090312 (+ 123us) server_negotiation.cc:407] Connection header received
0812 14:47:11.090388 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.090389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.090401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.090472 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:11.090975 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.091502 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.092225 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.092415 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.092502 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.094692 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:11.094693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:11.094693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:11.094700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:11.096231 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:11.096707 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:11.096708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:11.096709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:11.096772 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:11.096980 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:11.096981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:11.096982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:11.097105 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:11.097188 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.097406 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.097506 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 14:47:11.097783 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.089989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37545 (local address 127.0.0.1:45410)
0812 14:47:11.090211 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.090224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.090330 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.090478 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.090479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.090830 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:11.090876 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.090878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.091602 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.091603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.092135 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.092136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.092504 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.092505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.092570 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.093114 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:11.093118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:11.094581 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:11.096396 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:11.096405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:11.096407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:11.096623 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:11.096850 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:11.096850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:11.096851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:11.096915 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:11.097200 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:11.097235 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:11.097292 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.097379 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.097635 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":119,"thread_start_us":123,"threads_started":1}
I20260812 14:47:11.100065 17031 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45410:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:47:11.100306 17031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:11.102001 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 58f7d0f279004ede8f451f3dbfd07159 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6573fd79c866404ba70d3c121e245dd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:11.102051 17290 tablet_service.cc:1467] Processing CreateTablet for tablet 58f7d0f279004ede8f451f3dbfd07159 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6573fd79c866404ba70d3c121e245dd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:11.102198 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f7d0f279004ede8f451f3dbfd07159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.102254 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f7d0f279004ede8f451f3dbfd07159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.103440 17997 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: Bootstrap starting.
I20260812 14:47:11.103466 18004 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: Bootstrap starting.
I20260812 14:47:11.103578 17822 tablet_service.cc:1467] Processing CreateTablet for tablet 58f7d0f279004ede8f451f3dbfd07159 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6573fd79c866404ba70d3c121e245dd8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:11.103749 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f7d0f279004ede8f451f3dbfd07159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.103930 18004 tablet_bootstrap.cc:654] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.104137 17997 tablet_bootstrap.cc:654] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.104879 17997 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: No bootstrap required, opened a new log
I20260812 14:47:11.104897 18004 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: No bootstrap required, opened a new log
I20260812 14:47:11.104947 18004 ts_tablet_manager.cc:1397] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:11.104948 17997 ts_tablet_manager.cc:1397] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:11.105110 17997 raft_consensus.cc:348] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.105111 18004 raft_consensus.cc:348] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.105189 18004 raft_consensus.cc:374] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.105234 18004 raft_consensus.cc:729] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.105204 17997 raft_consensus.cc:374] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.105268 17997 raft_consensus.cc:729] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.105295 18004 consensus_queue.cc:260] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.105316 17997 consensus_queue.cc:260] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.105394 18004 ts_tablet_manager.cc:1428] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:11.105422 17997 ts_tablet_manager.cc:1428] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:11.105535 18001 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Bootstrap starting.
I20260812 14:47:11.105602 18010 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.105692 18010 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.106089 18001 tablet_bootstrap.cc:654] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.106849 18001 tablet_bootstrap.cc:492] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: No bootstrap required, opened a new log
I20260812 14:47:11.106915 18001 ts_tablet_manager.cc:1397] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:11.107065 18001 raft_consensus.cc:348] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.107142 18001 raft_consensus.cc:374] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.107175 18001 raft_consensus.cc:729] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.107227 18001 consensus_queue.cc:260] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.107311 18001 ts_tablet_manager.cc:1428] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:11.107389 18011 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.107445 18014 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.107517 18011 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.107548 18014 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.147872 18010 raft_consensus.cc:482] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:11.147974 18010 raft_consensus.cc:504] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.148458 18010 leader_election.cc:288] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:11.151727 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.149018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:53057 (local address 127.13.146.131:34335)
0812 14:47:11.149061 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.149063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.149071 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:11.149152 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.149153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.149161 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.149269 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.149448 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.149916 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.150923 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.151342 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.151419 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.151434 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.151604 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.151652 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:11.151793 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.148740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:56391 (local address 127.13.146.129:40789)
0812 14:47:11.148804 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.148806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.148964 (+ 158us) server_negotiation.cc:407] Connection header received
0812 14:47:11.149037 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.149038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.149050 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.149163 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.149371 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.149896 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.150972 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.151379 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.151480 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.151489 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.151658 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.151703 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:47:11.151816 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.148900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:34335 (local address 127.13.146.130:53057)
0812 14:47:11.149003 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.149015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.149068 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.149277 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.149278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.149280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.149372 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.149374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.150000 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.150001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.150823 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.150825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.151440 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.151441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.151519 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.151523 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.151589 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.151654 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":34,"thread_start_us":53,"threads_started":1}
I20260812 14:47:11.151821 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.148658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:40789 (local address 127.13.146.130:56391)
0812 14:47:11.148878 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.148889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.148975 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.149168 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.149169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.149172 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.149257 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.149259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.149987 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.149988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.150868 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.150870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.151490 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.151491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.151570 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.151575 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.151641 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.151680 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":29,"thread_start_us":89,"threads_started":1}
I20260812 14:47:11.152048 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f7d0f279004ede8f451f3dbfd07159" candidate_uuid: "59e34a2e80f1446ebf8651fd49167249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" is_pre_election: true
I20260812 14:47:11.152146 17858 raft_consensus.cc:2454] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e34a2e80f1446ebf8651fd49167249 in term 0.
I20260812 14:47:11.152339 17470 leader_election.cc:302] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48725a06e6fd4e6abf7aa4cf1680b0cd, 59e34a2e80f1446ebf8651fd49167249; no voters:
I20260812 14:47:11.152369 17328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f7d0f279004ede8f451f3dbfd07159" candidate_uuid: "59e34a2e80f1446ebf8651fd49167249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c" is_pre_election: true
I20260812 14:47:11.152483 17328 raft_consensus.cc:2454] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59e34a2e80f1446ebf8651fd49167249 in term 0.
I20260812 14:47:11.152516 18010 raft_consensus.cc:2781] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:11.152565 18010 raft_consensus.cc:482] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:11.152586 18010 raft_consensus.cc:3037] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.153330 18010 raft_consensus.cc:504] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.153445 18010 leader_election.cc:288] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [CANDIDATE]: Term 1 election: Requested vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:11.153685 17328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f7d0f279004ede8f451f3dbfd07159" candidate_uuid: "59e34a2e80f1446ebf8651fd49167249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c"
I20260812 14:47:11.153774 17328 raft_consensus.cc:3037] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.153724 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f7d0f279004ede8f451f3dbfd07159" candidate_uuid: "59e34a2e80f1446ebf8651fd49167249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
I20260812 14:47:11.153827 17858 raft_consensus.cc:3037] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.154466 17328 raft_consensus.cc:2454] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e34a2e80f1446ebf8651fd49167249 in term 1.
I20260812 14:47:11.154641 17858 raft_consensus.cc:2454] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59e34a2e80f1446ebf8651fd49167249 in term 1.
I20260812 14:47:11.154641 17470 leader_election.cc:302] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 59e34a2e80f1446ebf8651fd49167249; no voters:
I20260812 14:47:11.154863 18010 raft_consensus.cc:2781] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:11.155025 18010 raft_consensus.cc:686] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [term 1 LEADER]: Becoming Leader. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Running, Role: LEADER
I20260812 14:47:11.155138 18010 consensus_queue.cc:237] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.155604 18055 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: TxnStatusTablet state changed. Reason: New leader 59e34a2e80f1446ebf8651fd49167249. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.155709 18055 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:11.155906 18059 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:47:11.156054 18059 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:47:11.155999 17032 catalog_manager.cc:5495] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130). New cstate: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:11.166103 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.163547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.13.146.130:42293)
0812 14:47:11.163597 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.163598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.163615 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:47:11.163691 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.163692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.163710 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.163775 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.163904 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.164486 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.165261 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.165462 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.165541 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.165752 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.165857 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.165943 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.166004 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.166025 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:47:11.166136 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.163465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.0.0.1:34962)
0812 14:47:11.163544 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.163556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.163607 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.163780 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.163781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.163782 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:11.163834 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.163836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.164571 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.164572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.165136 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.165138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.165539 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.165540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.165613 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.165640 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.165934 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.165945 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.165998 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.166054 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:47:11.168156 17858 raft_consensus.cc:1261] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Refusing update from remote peer 59e34a2e80f1446ebf8651fd49167249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.168192 17328 raft_consensus.cc:1261] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Refusing update from remote peer 59e34a2e80f1446ebf8651fd49167249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.168576 18055 consensus_queue.cc:1035] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.168731 18010 consensus_queue.cc:1035] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.170083 18014 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: TxnStatusTablet state changed. Reason: New leader 59e34a2e80f1446ebf8651fd49167249. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.170179 18014 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.170408 18011 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: TxnStatusTablet state changed. Reason: New leader 59e34a2e80f1446ebf8651fd49167249. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.170495 18011 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.171485 18055 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.171583 18055 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:11.171973 18010 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.172050 18010 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 59e34a2e80f1446ebf8651fd49167249: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:11.172605 18065 mvcc.cc:205] Tried to move back new op lower bound from 7317692543662100480 to 7317692543612059648. Current Snapshot: MvccSnapshot[applied={T|T < 7317692543662100480}]
I20260812 14:47:11.172609 18011 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.173238 18011 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 48725a06e6fd4e6abf7aa4cf1680b0cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.173648 18064 mvcc.cc:205] Tried to move back new op lower bound from 7317692543662100480 to 7317692543612059648. Current Snapshot: MvccSnapshot[applied={T|T < 7317692543662100480 or (T in {7317692543662100480})}]
I20260812 14:47:11.174060 18070 mvcc.cc:205] Tried to move back new op lower bound from 7317692543662100480 to 7317692543612059648. Current Snapshot: MvccSnapshot[applied={T|T < 7317692543662100480 or (T in {7317692543662100480})}]
I20260812 14:47:11.172814 18014 tablet_replica.cc:439] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59e34a2e80f1446ebf8651fd49167249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } } }
I20260812 14:47:11.174738 18014 tablet_replica.cc:442] T 58f7d0f279004ede8f451f3dbfd07159 P 2a6ec476acb6478987ca782cbc968b3c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:11.180500 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.177612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.13.146.129:40789)
0812 14:47:11.177666 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.177668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.177729 (+ 61us) server_negotiation.cc:407] Connection header received
0812 14:47:11.177816 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.177816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.177833 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.177903 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.178061 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.178554 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.179688 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.179930 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.179983 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.180201 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.180298 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.180365 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.180432 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.180448 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:11.186070 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.183443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:56903 (local address 127.13.146.130:42293)
0812 14:47:11.183493 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.183495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.183625 (+ 130us) server_negotiation.cc:407] Connection header received
0812 14:47:11.183692 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.183693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.183703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.183753 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.183904 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.184376 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.185168 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.185490 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.185584 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.185812 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.185851 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.185912 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.185975 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.185996 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:47:11.186120 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.183363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.13.146.129:56903)
0812 14:47:11.183541 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.183551 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.183627 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.183753 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.183754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.183755 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:11.183821 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.183822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.184463 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.184464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.185069 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.185071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.185586 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.185587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.185654 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.185668 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.185914 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.185917 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.185972 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.186015 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260812 14:47:11.188844 17858 raft_consensus.cc:1261] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Refusing update from remote peer 2a6ec476acb6478987ca782cbc968b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.189075 17598 raft_consensus.cc:1261] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Refusing update from remote peer 2a6ec476acb6478987ca782cbc968b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.189147 18014 consensus_queue.cc:1035] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.189392 18021 consensus_queue.cc:1035] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.198758 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.196021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:52721 (local address 127.13.146.129:40789)
0812 14:47:11.196146 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.196148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.196156 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:11.196178 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.196179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.196189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.196255 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.196385 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.196873 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.197947 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.198226 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.198326 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.198516 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.198548 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.198598 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.198664 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.198684 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 14:47:11.198798 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.195695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:40789 (local address 127.13.146.130:52721)
0812 14:47:11.195933 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.195946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.196001 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.196257 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.196258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.196259 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:11.196312 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.196313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.196966 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.196967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.197838 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.197843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.198328 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.198329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.198413 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.198428 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.198608 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.198610 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.198660 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.198700 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260812 14:47:11.212423 17031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:11.212548 17031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:11.213442 17031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Sending DeleteTablet for 3 replicas of tablet 9adac297c23c4c3687a90c168d221b07
I20260812 14:47:11.213811 17822 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adac297c23c4c3687a90c168d221b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:11 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:58472
I20260812 14:47:11.214068 17290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adac297c23c4c3687a90c168d221b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:11 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56778
I20260812 14:47:11.214090 18106 tablet_replica.cc:330] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: stopping tablet replica
I20260812 14:47:11.214211 18106 raft_consensus.cc:2229] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:11.214264 18106 raft_consensus.cc:2258] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:11.214798 17031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:11.214944 17031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:11.216174 17562 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adac297c23c4c3687a90c168d221b07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:11 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:34958
I20260812 14:47:11.216507 18110 tablet_replica.cc:330] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: stopping tablet replica
I20260812 14:47:11.216531 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c (DEFAULT_TABLE table=test-table [id=f3d2de3eab0641428f414efdef6c2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:11.216676 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ea4bff01c240e4b1a6fdd955a5060c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.216696 18110 raft_consensus.cc:2229] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:11.216912 18110 raft_consensus.cc:2258] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:11.217438 17822 tablet_service.cc:1467] Processing CreateTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c (DEFAULT_TABLE table=test-table [id=f3d2de3eab0641428f414efdef6c2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:11.217563 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ea4bff01c240e4b1a6fdd955a5060c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.218075 18106 ts_tablet_manager.cc:1905] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:11.218220 17997 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Bootstrap starting.
I20260812 14:47:11.218649 17997 tablet_bootstrap.cc:654] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.219082 18001 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Bootstrap starting.
I20260812 14:47:11.219301 17290 tablet_service.cc:1467] Processing CreateTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c (DEFAULT_TABLE table=test-table [id=f3d2de3eab0641428f414efdef6c2e16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:11.219413 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ea4bff01c240e4b1a6fdd955a5060c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:11.219501 18001 tablet_bootstrap.cc:654] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.219727 18106 ts_tablet_manager.cc:1918] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:11.219802 18106 log.cc:1196] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/9adac297c23c4c3687a90c168d221b07
I20260812 14:47:11.220062 18106 ts_tablet_manager.cc:1939] T 9adac297c23c4c3687a90c168d221b07 P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting consensus metadata
I20260812 14:47:11.220471 17001 catalog_manager.cc:4841] TS 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335): tablet 9adac297c23c4c3687a90c168d221b07 (table test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]) successfully deleted
I20260812 14:47:11.220834 18110 ts_tablet_manager.cc:1905] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:11.220960 18004 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Bootstrap starting.
I20260812 14:47:11.221412 18004 tablet_bootstrap.cc:654] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:11.221665 18001 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: No bootstrap required, opened a new log
I20260812 14:47:11.221712 18001 ts_tablet_manager.cc:1397] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:47:11.221848 18001 raft_consensus.cc:348] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.221911 18001 raft_consensus.cc:374] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.221931 18001 raft_consensus.cc:729] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.221972 18001 consensus_queue.cc:260] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.222064 18001 ts_tablet_manager.cc:1428] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:11.222735 17997 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: No bootstrap required, opened a new log
I20260812 14:47:11.222787 17997 ts_tablet_manager.cc:1397] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 14:47:11.222944 17997 raft_consensus.cc:348] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.223016 17997 raft_consensus.cc:374] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.223031 17997 raft_consensus.cc:729] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.223088 17997 consensus_queue.cc:260] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.223198 17997 ts_tablet_manager.cc:1428] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:11.224874 18110 ts_tablet_manager.cc:1918] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:11.224967 18110 log.cc:1196] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/9adac297c23c4c3687a90c168d221b07
I20260812 14:47:11.225268 18110 ts_tablet_manager.cc:1939] T 9adac297c23c4c3687a90c168d221b07 P 2a6ec476acb6478987ca782cbc968b3c: Deleting consensus metadata
I20260812 14:47:11.225391 18111 tablet_replica.cc:330] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: stopping tablet replica
I20260812 14:47:11.225600 18111 raft_consensus.cc:2229] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:11.225708 18111 raft_consensus.cc:2258] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:11.225720 17001 catalog_manager.cc:4841] TS 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789): tablet 9adac297c23c4c3687a90c168d221b07 (table test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]) successfully deleted
I20260812 14:47:11.226258 18111 ts_tablet_manager.cc:1905] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:11.227872 18111 ts_tablet_manager.cc:1918] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:11.227953 18111 log.cc:1196] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/9adac297c23c4c3687a90c168d221b07
I20260812 14:47:11.228228 18111 ts_tablet_manager.cc:1939] T 9adac297c23c4c3687a90c168d221b07 P 59e34a2e80f1446ebf8651fd49167249: Deleting consensus metadata
I20260812 14:47:11.228690 17003 catalog_manager.cc:4841] TS 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293): tablet 9adac297c23c4c3687a90c168d221b07 (table test-table [id=096a1bbb8b434d2ba9745fb4519b5c15]) successfully deleted
I20260812 14:47:11.228847 18004 tablet_bootstrap.cc:492] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: No bootstrap required, opened a new log
I20260812 14:47:11.228902 18004 ts_tablet_manager.cc:1397] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260812 14:47:11.229048 18004 raft_consensus.cc:348] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.229125 18004 raft_consensus.cc:374] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:11.229146 18004 raft_consensus.cc:729] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Initialized, Role: FOLLOWER
I20260812 14:47:11.229189 18004 consensus_queue.cc:260] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.229302 18004 ts_tablet_manager.cc:1428] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:11.234438 18011 raft_consensus.cc:482] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:11.234513 18011 raft_consensus.cc:504] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.234835 18011 leader_election.cc:288] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293)
I20260812 14:47:11.238741 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.234989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:40789 (local address 127.13.146.131:36697)
0812 14:47:11.235137 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.235149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.235195 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.235406 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.235407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.235409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.235476 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.235479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.236386 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.236388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.237696 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.237699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.238472 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.238473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.238555 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.238558 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.238618 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.238653 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260812 14:47:11.239259 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.235088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:36697 (local address 127.13.146.129:40789)
0812 14:47:11.235141 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.235144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.235308 (+ 164us) server_negotiation.cc:407] Connection header received
0812 14:47:11.235334 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.235335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.235346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.235394 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.235566 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.236293 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.237795 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.238364 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.238431 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.238440 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.239158 (+ 718us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.239195 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:47:11.239446 17328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ea4bff01c240e4b1a6fdd955a5060c" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c" is_pre_election: true
I20260812 14:47:11.239485 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.235337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:36135 (local address 127.13.146.130:42293)
0812 14:47:11.235379 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.235381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.235782 (+ 401us) server_negotiation.cc:407] Connection header received
0812 14:47:11.235871 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.235872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.235881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.235908 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:11.236438 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.237148 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.238586 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.239136 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.239178 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.239185 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.239392 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.239422 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:47:11.239530 17328 raft_consensus.cc:2454] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 0.
I20260812 14:47:11.239719 17729 leader_election.cc:302] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 48725a06e6fd4e6abf7aa4cf1680b0cd; no voters:
I20260812 14:47:11.239732 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.235040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.13.146.131:36135)
0812 14:47:11.235724 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.235734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.235805 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.235963 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.235963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.235965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:11.236349 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.236351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.237299 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.237300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.238466 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.238469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.239240 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.239241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.239319 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.239323 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.239602 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.239642 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260812 14:47:11.239898 18011 raft_consensus.cc:2781] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:11.239939 18011 raft_consensus.cc:482] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:11.239967 18011 raft_consensus.cc:3037] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.240036 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ea4bff01c240e4b1a6fdd955a5060c" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249" is_pre_election: true
I20260812 14:47:11.240120 17598 raft_consensus.cc:2454] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 0.
I20260812 14:47:11.240792 18011 raft_consensus.cc:504] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.240926 18011 leader_election.cc:288] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 election: Requested vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293)
I20260812 14:47:11.241096 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ea4bff01c240e4b1a6fdd955a5060c" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249"
I20260812 14:47:11.241171 17598 raft_consensus.cc:3037] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.241168 17328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ea4bff01c240e4b1a6fdd955a5060c" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c"
I20260812 14:47:11.241261 17328 raft_consensus.cc:3037] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:11.241835 17328 raft_consensus.cc:2454] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 1.
I20260812 14:47:11.241891 17598 raft_consensus.cc:2454] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 1.
I20260812 14:47:11.242008 17729 leader_election.cc:302] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 48725a06e6fd4e6abf7aa4cf1680b0cd; no voters:
I20260812 14:47:11.242084 18011 raft_consensus.cc:2781] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:11.242236 18011 raft_consensus.cc:686] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 LEADER]: Becoming Leader. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Running, Role: LEADER
I20260812 14:47:11.242309 18011 consensus_queue.cc:237] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:11.243011 17031 catalog_manager.cc:5495] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131). New cstate: current_term: 1 leader_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:11.254169 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.250992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58482 (local address 127.13.146.131:34335)
0812 14:47:11.251032 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.251034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.251043 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:11.251127 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.251127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.251147 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.251223 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.251370 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.252124 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.253246 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.253558 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.253632 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.253807 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.253932 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.254009 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.254064 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.254088 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:11.261252 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.258156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42293 (local address 127.13.146.131:43223)
0812 14:47:11.258232 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.258243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.258280 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.258408 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.258409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.258410 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:11.258470 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.258472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.259308 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.259309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.260226 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.260228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.260746 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.260747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.260816 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.260832 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.261035 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.261037 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.261085 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.261135 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 14:47:11.261252 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.258221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:43223 (local address 127.13.146.130:42293)
0812 14:47:11.258265 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.258267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.258277 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:11.258334 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.258335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.258345 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.258407 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.258544 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.259214 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.260318 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.260650 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.260720 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.260923 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.260961 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.261025 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.261089 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.261113 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:11.264101 17598 raft_consensus.cc:1261] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Refusing update from remote peer 48725a06e6fd4e6abf7aa4cf1680b0cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.264142 17328 raft_consensus.cc:1261] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Refusing update from remote peer 48725a06e6fd4e6abf7aa4cf1680b0cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:11.264374 18118 consensus_queue.cc:1035] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.264482 18011 consensus_queue.cc:1035] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:11.274827 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.271895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42729 (local address 127.13.146.131:34335)
0812 14:47:11.271932 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:11.271934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:11.271941 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:47:11.271960 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:11.271961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:11.271970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:11.272015 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:11.272174 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.272888 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.274031 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.274324 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.274387 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.274583 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.274616 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.274669 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:11.274739 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:47:11.274756 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:11.274859 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:11.271680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:34335 (local address 127.13.146.130:42729)
0812 14:47:11.271774 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:11.271786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:11.271828 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:11.272057 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:11.272058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:11.272059 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:11.272101 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.272102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.272980 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.272981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.273919 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:11.273922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:11.274415 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:11.274416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:11.274487 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:11.274495 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:11.274685 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:11.274687 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:11.274738 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:47:11.274783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 14:47:11.275136 18065 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:47:12.808593 17032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:12.808710 17032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:12.809685 17032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Sending DeleteTablet for 3 replicas of tablet 06ea4bff01c240e4b1a6fdd955a5060c
I20260812 14:47:12.809973 17822 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:58472
I20260812 14:47:12.809988 17290 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56778
I20260812 14:47:12.809979 17562 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ea4bff01c240e4b1a6fdd955a5060c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:34958
I20260812 14:47:12.810235 18186 tablet_replica.cc:330] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: stopping tablet replica
I20260812 14:47:12.810237 18185 tablet_replica.cc:330] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: stopping tablet replica
I20260812 14:47:12.810302 18185 raft_consensus.cc:2229] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:12.810302 18186 raft_consensus.cc:2229] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:12.810338 18185 raft_consensus.cc:2258] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:12.810451 18184 tablet_replica.cc:330] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: stopping tablet replica
I20260812 14:47:12.810515 18184 raft_consensus.cc:2229] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:12.810518 18186 raft_consensus.cc:2258] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:12.810562 18184 raft_consensus.cc:2258] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:12.810787 18185 ts_tablet_manager.cc:1905] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:12.810786 18186 ts_tablet_manager.cc:1905] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:12.812573 18185 ts_tablet_manager.cc:1918] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:47:12.812655 18185 log.cc:1196] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/06ea4bff01c240e4b1a6fdd955a5060c
I20260812 14:47:12.812702 18184 ts_tablet_manager.cc:1905] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:12.812891 18185 ts_tablet_manager.cc:1939] T 06ea4bff01c240e4b1a6fdd955a5060c P 59e34a2e80f1446ebf8651fd49167249: Deleting consensus metadata
I20260812 14:47:12.813063 18186 ts_tablet_manager.cc:1918] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:47:12.813120 18186 log.cc:1196] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/06ea4bff01c240e4b1a6fdd955a5060c
I20260812 14:47:12.813387 18186 ts_tablet_manager.cc:1939] T 06ea4bff01c240e4b1a6fdd955a5060c P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting consensus metadata
I20260812 14:47:12.813728 17001 catalog_manager.cc:4841] TS 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335): tablet 06ea4bff01c240e4b1a6fdd955a5060c (table test-table [id=f3d2de3eab0641428f414efdef6c2e16]) successfully deleted
I20260812 14:47:12.814082 17003 catalog_manager.cc:4841] TS 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293): tablet 06ea4bff01c240e4b1a6fdd955a5060c (table test-table [id=f3d2de3eab0641428f414efdef6c2e16]) successfully deleted
I20260812 14:47:12.814512 18184 ts_tablet_manager.cc:1918] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:47:12.814563 18184 log.cc:1196] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/06ea4bff01c240e4b1a6fdd955a5060c
I20260812 14:47:12.814805 18184 ts_tablet_manager.cc:1939] T 06ea4bff01c240e4b1a6fdd955a5060c P 2a6ec476acb6478987ca782cbc968b3c: Deleting consensus metadata
I20260812 14:47:12.815256 17001 catalog_manager.cc:4841] TS 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789): tablet 06ea4bff01c240e4b1a6fdd955a5060c (table test-table [id=f3d2de3eab0641428f414efdef6c2e16]) successfully deleted
Aug 12 14:47:12 dist-test-slave-8hhm krb5kdc[16949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546032, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:47:12 dist-test-slave-8hhm krb5kdc[16949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546032, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:12.835704 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:12.828516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50120 (local address 127.13.146.190:37545)
0812 14:47:12.828677 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:12.828679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:12.828751 (+ 72us) server_negotiation.cc:407] Connection header received
0812 14:47:12.828789 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:12.828790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:12.828801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:12.828873 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:12.829500 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:12.830039 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:12.830747 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:12.830958 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:12.831011 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:12.833316 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:12.833318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:12.833319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:12.833327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:12.834740 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:12.835062 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:12.835063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:12.835063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:12.835121 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:12.835290 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:12.835290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:12.835291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:12.835393 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:12.835472 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:12.835556 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:47:12.835632 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260812 14:47:12.837255 17032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:12.837419 17032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:12.838817 17822 tablet_service.cc:1467] Processing CreateTablet for tablet 44608b71daa6442393b280fbc380fbab (DEFAULT_TABLE table=test-table [id=870785fd338248499fac41219dc5ca88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:12.838832 17290 tablet_service.cc:1467] Processing CreateTablet for tablet 44608b71daa6442393b280fbc380fbab (DEFAULT_TABLE table=test-table [id=870785fd338248499fac41219dc5ca88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:12.838866 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 44608b71daa6442393b280fbc380fbab (DEFAULT_TABLE table=test-table [id=870785fd338248499fac41219dc5ca88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:12.838990 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44608b71daa6442393b280fbc380fbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:12.838992 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44608b71daa6442393b280fbc380fbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:12.838994 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44608b71daa6442393b280fbc380fbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:12.840085 18201 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Bootstrap starting.
I20260812 14:47:12.840410 18202 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Bootstrap starting.
I20260812 14:47:12.840643 18201 tablet_bootstrap.cc:654] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:12.840703 18203 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Bootstrap starting.
I20260812 14:47:12.841032 18202 tablet_bootstrap.cc:654] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:12.841137 18203 tablet_bootstrap.cc:654] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:12.841423 18201 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: No bootstrap required, opened a new log
I20260812 14:47:12.841492 18201 ts_tablet_manager.cc:1397] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:12.841650 18201 raft_consensus.cc:348] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.841743 18201 raft_consensus.cc:374] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:12.841742 18202 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: No bootstrap required, opened a new log
I20260812 14:47:12.841768 18201 raft_consensus.cc:729] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Initialized, Role: FOLLOWER
I20260812 14:47:12.841786 18202 ts_tablet_manager.cc:1397] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:12.841838 18201 consensus_queue.cc:260] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.841981 18201 ts_tablet_manager.cc:1428] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:12.841954 18202 raft_consensus.cc:348] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.842023 18202 raft_consensus.cc:374] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:12.842057 18202 raft_consensus.cc:729] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Initialized, Role: FOLLOWER
I20260812 14:47:12.842127 18202 consensus_queue.cc:260] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.842322 18202 ts_tablet_manager.cc:1428] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:12.843031 18203 tablet_bootstrap.cc:492] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: No bootstrap required, opened a new log
I20260812 14:47:12.843096 18203 ts_tablet_manager.cc:1397] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:12.843231 18203 raft_consensus.cc:348] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.843315 18203 raft_consensus.cc:374] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:12.843335 18203 raft_consensus.cc:729] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Initialized, Role: FOLLOWER
I20260812 14:47:12.843401 18203 consensus_queue.cc:260] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:12.843503 18203 ts_tablet_manager.cc:1428] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:13.017802 18174 raft_consensus.cc:482] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:13.017921 18174 raft_consensus.cc:504] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.018175 18174 leader_election.cc:288] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293)
I20260812 14:47:13.018460 17327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44608b71daa6442393b280fbc380fbab" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c" is_pre_election: true
I20260812 14:47:13.018462 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44608b71daa6442393b280fbc380fbab" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249" is_pre_election: true
I20260812 14:47:13.018573 17327 raft_consensus.cc:2454] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 0.
I20260812 14:47:13.018595 17598 raft_consensus.cc:2454] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 0.
I20260812 14:47:13.018790 17731 leader_election.cc:302] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48725a06e6fd4e6abf7aa4cf1680b0cd, 59e34a2e80f1446ebf8651fd49167249; no voters:
I20260812 14:47:13.018916 18174 raft_consensus.cc:2781] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:13.018973 18174 raft_consensus.cc:482] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:13.019008 18174 raft_consensus.cc:3037] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.019773 18174 raft_consensus.cc:504] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.019933 18174 leader_election.cc:288] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 election: Requested vote from peers 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789), 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293)
I20260812 14:47:13.020135 17327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44608b71daa6442393b280fbc380fbab" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6ec476acb6478987ca782cbc968b3c"
I20260812 14:47:13.020113 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44608b71daa6442393b280fbc380fbab" candidate_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249"
I20260812 14:47:13.020215 17327 raft_consensus.cc:3037] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.020239 17598 raft_consensus.cc:3037] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.021134 17327 raft_consensus.cc:2454] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 1.
I20260812 14:47:13.021173 17598 raft_consensus.cc:2454] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48725a06e6fd4e6abf7aa4cf1680b0cd in term 1.
I20260812 14:47:13.021334 17729 leader_election.cc:302] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 48725a06e6fd4e6abf7aa4cf1680b0cd; no voters:
I20260812 14:47:13.021445 18174 raft_consensus.cc:2781] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:13.021533 18174 raft_consensus.cc:686] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 LEADER]: Becoming Leader. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Running, Role: LEADER
I20260812 14:47:13.021590 18174 consensus_queue.cc:237] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.022318 17032 catalog_manager.cc:5495] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131). New cstate: current_term: 1 leader_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:13.072904 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:13.070450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.13.146.131:34335)
0812 14:47:13.070640 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:13.070642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:13.070655 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:13.070679 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:13.070679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:13.070690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:13.070753 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:13.070938 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:13.071420 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:13.072212 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:13.072435 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:13.072517 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:13.072629 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:13.072659 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:13.072698 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:13.072752 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 14:47:13.072772 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
I20260812 14:47:13.075599 17598 raft_consensus.cc:1261] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Refusing update from remote peer 48725a06e6fd4e6abf7aa4cf1680b0cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:13.075614 17327 raft_consensus.cc:1261] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Refusing update from remote peer 48725a06e6fd4e6abf7aa4cf1680b0cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:13.075863 18174 consensus_queue.cc:1035] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:13.075996 18165 consensus_queue.cc:1035] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:13.087476 17032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:13.087604 17032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:13.088271 17032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Sending DeleteTablet for 3 replicas of tablet 44608b71daa6442393b280fbc380fbab
I20260812 14:47:13.088511 17562 tablet_service.cc:1514] Processing DeleteTablet for tablet 44608b71daa6442393b280fbc380fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:13 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:34958
I20260812 14:47:13.088550 17290 tablet_service.cc:1514] Processing DeleteTablet for tablet 44608b71daa6442393b280fbc380fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:13 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56778
I20260812 14:47:13.088589 17822 tablet_service.cc:1514] Processing DeleteTablet for tablet 44608b71daa6442393b280fbc380fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:13 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:58472
I20260812 14:47:13.088675 18185 tablet_replica.cc:330] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: stopping tablet replica
I20260812 14:47:13.088778 18185 raft_consensus.cc:2229] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:13.088832 18185 raft_consensus.cc:2258] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:13.088786 17032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:47:13.088912 18184 tablet_replica.cc:330] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: stopping tablet replica
W20260812 14:47:13.088948 17032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:13.088975 18184 raft_consensus.cc:2229] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:13.089071 18184 raft_consensus.cc:2258] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:13.089171 18186 tablet_replica.cc:330] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: stopping tablet replica
I20260812 14:47:13.089253 18186 raft_consensus.cc:2229] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:13.089316 18186 raft_consensus.cc:2258] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:13.089676 18184 ts_tablet_manager.cc:1905] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:13.089712 18186 ts_tablet_manager.cc:1905] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:13.089811 18185 ts_tablet_manager.cc:1905] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:13.090317 17290 tablet_service.cc:1467] Processing CreateTablet for tablet 8961b35811b54a61be9ed425a2e304ef (DEFAULT_TABLE table=test-table [id=72c471ffc0ce424ab22e35ac349bf804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:13.090428 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8961b35811b54a61be9ed425a2e304ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:13.091498 18186 ts_tablet_manager.cc:1918] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:13.091559 18186 log.cc:1196] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/44608b71daa6442393b280fbc380fbab
I20260812 14:47:13.091794 18186 ts_tablet_manager.cc:1939] T 44608b71daa6442393b280fbc380fbab P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting consensus metadata
I20260812 14:47:13.091886 18202 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c: Bootstrap starting.
I20260812 14:47:13.092172 18185 ts_tablet_manager.cc:1918] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:13.092239 18185 log.cc:1196] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/44608b71daa6442393b280fbc380fbab
I20260812 14:47:13.092227 17001 catalog_manager.cc:4841] TS 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335): tablet 44608b71daa6442393b280fbc380fbab (table test-table [id=870785fd338248499fac41219dc5ca88]) successfully deleted
I20260812 14:47:13.092272 17822 tablet_service.cc:1467] Processing CreateTablet for tablet 8961b35811b54a61be9ed425a2e304ef (DEFAULT_TABLE table=test-table [id=72c471ffc0ce424ab22e35ac349bf804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:13.092309 18202 tablet_bootstrap.cc:654] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:13.092402 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8961b35811b54a61be9ed425a2e304ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:13.092484 18185 ts_tablet_manager.cc:1939] T 44608b71daa6442393b280fbc380fbab P 59e34a2e80f1446ebf8651fd49167249: Deleting consensus metadata
I20260812 14:47:13.092701 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 8961b35811b54a61be9ed425a2e304ef (DEFAULT_TABLE table=test-table [id=72c471ffc0ce424ab22e35ac349bf804]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:13.092782 18184 ts_tablet_manager.cc:1918] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:13.092799 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8961b35811b54a61be9ed425a2e304ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:13.092828 18184 log.cc:1196] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/44608b71daa6442393b280fbc380fbab
I20260812 14:47:13.092869 18202 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c: No bootstrap required, opened a new log
I20260812 14:47:13.092922 18202 ts_tablet_manager.cc:1397] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:13.093039 18184 ts_tablet_manager.cc:1939] T 44608b71daa6442393b280fbc380fbab P 2a6ec476acb6478987ca782cbc968b3c: Deleting consensus metadata
I20260812 14:47:13.093065 18202 raft_consensus.cc:348] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.093150 18202 raft_consensus.cc:374] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:13.093180 18202 raft_consensus.cc:729] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Initialized, Role: FOLLOWER
I20260812 14:47:13.093268 18202 consensus_queue.cc:260] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.093379 18202 ts_tablet_manager.cc:1428] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:13.093789 18201 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Bootstrap starting.
I20260812 14:47:13.094157 18201 tablet_bootstrap.cc:654] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:13.094208 18203 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Bootstrap starting.
I20260812 14:47:13.094336 17001 catalog_manager.cc:4841] TS 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789): tablet 44608b71daa6442393b280fbc380fbab (table test-table [id=870785fd338248499fac41219dc5ca88]) successfully deleted
I20260812 14:47:13.094498 17003 catalog_manager.cc:4841] TS 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293): tablet 44608b71daa6442393b280fbc380fbab (table test-table [id=870785fd338248499fac41219dc5ca88]) successfully deleted
I20260812 14:47:13.094564 18203 tablet_bootstrap.cc:654] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:13.094928 18201 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: No bootstrap required, opened a new log
I20260812 14:47:13.094965 18201 ts_tablet_manager.cc:1397] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:13.095054 18201 raft_consensus.cc:348] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.095103 18201 raft_consensus.cc:374] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:13.095119 18201 raft_consensus.cc:729] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48725a06e6fd4e6abf7aa4cf1680b0cd, State: Initialized, Role: FOLLOWER
I20260812 14:47:13.095121 18203 tablet_bootstrap.cc:492] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: No bootstrap required, opened a new log
I20260812 14:47:13.095156 18203 ts_tablet_manager.cc:1397] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:13.095186 18201 consensus_queue.cc:260] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.095288 18201 ts_tablet_manager.cc:1428] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:13.095244 18203 raft_consensus.cc:348] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.095328 18203 raft_consensus.cc:374] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:13.095348 18203 raft_consensus.cc:729] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e34a2e80f1446ebf8651fd49167249, State: Initialized, Role: FOLLOWER
I20260812 14:47:13.095391 18203 consensus_queue.cc:260] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.095505 18203 ts_tablet_manager.cc:1428] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:13.235124 18209 raft_consensus.cc:482] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:13.235242 18209 raft_consensus.cc:504] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.235468 18209 leader_election.cc:288] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:13.235736 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8961b35811b54a61be9ed425a2e304ef" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249" is_pre_election: true
I20260812 14:47:13.235737 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8961b35811b54a61be9ed425a2e304ef" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" is_pre_election: true
I20260812 14:47:13.235836 17598 raft_consensus.cc:2454] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 0.
I20260812 14:47:13.235863 17858 raft_consensus.cc:2454] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 0.
I20260812 14:47:13.236016 17203 leader_election.cc:302] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 59e34a2e80f1446ebf8651fd49167249; no voters:
I20260812 14:47:13.236166 18209 raft_consensus.cc:2781] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:13.236224 18209 raft_consensus.cc:482] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:13.236240 18209 raft_consensus.cc:3037] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.237010 18209 raft_consensus.cc:504] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.237169 18209 leader_election.cc:288] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 election: Requested vote from peers 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293), 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335)
I20260812 14:47:13.237393 17598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8961b35811b54a61be9ed425a2e304ef" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e34a2e80f1446ebf8651fd49167249"
I20260812 14:47:13.237393 17858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8961b35811b54a61be9ed425a2e304ef" candidate_uuid: "2a6ec476acb6478987ca782cbc968b3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd"
I20260812 14:47:13.237483 17858 raft_consensus.cc:3037] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.237483 17598 raft_consensus.cc:3037] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:13.238282 17858 raft_consensus.cc:2454] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 1.
I20260812 14:47:13.238299 17598 raft_consensus.cc:2454] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6ec476acb6478987ca782cbc968b3c in term 1.
I20260812 14:47:13.238458 17198 leader_election.cc:302] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6ec476acb6478987ca782cbc968b3c, 48725a06e6fd4e6abf7aa4cf1680b0cd; no voters:
I20260812 14:47:13.238612 18209 raft_consensus.cc:2781] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:13.238747 18209 raft_consensus.cc:686] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [term 1 LEADER]: Becoming Leader. State: Replica: 2a6ec476acb6478987ca782cbc968b3c, State: Running, Role: LEADER
I20260812 14:47:13.238809 18209 consensus_queue.cc:237] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } }
I20260812 14:47:13.239487 17032 catalog_manager.cc:5495] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129). New cstate: current_term: 1 leader_uuid: "2a6ec476acb6478987ca782cbc968b3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6ec476acb6478987ca782cbc968b3c" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 40789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:13.280503 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:13.278264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.13.146.129:40789)
0812 14:47:13.278449 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:13.278450 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:13.278457 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:47:13.278476 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:13.278477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:13.278486 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:13.278544 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:13.278662 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:13.279139 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:13.279816 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:13.280030 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:13.280079 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:13.280251 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:13.280278 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:13.280317 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:13.280421 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:47:13.280439 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260812 14:47:13.282774 17598 raft_consensus.cc:1261] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Refusing update from remote peer 2a6ec476acb6478987ca782cbc968b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:13.282763 17858 raft_consensus.cc:1261] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Refusing update from remote peer 2a6ec476acb6478987ca782cbc968b3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:13.283027 18229 consensus_queue.cc:1035] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "48725a06e6fd4e6abf7aa4cf1680b0cd" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:13.283136 18209 consensus_queue.cc:1035] T 8961b35811b54a61be9ed425a2e304ef P 2a6ec476acb6478987ca782cbc968b3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e34a2e80f1446ebf8651fd49167249" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:47:13.287765 18151 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:47:14.806154 17029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:14.806280 17029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:14.807590 17029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e710dba17be4458dac126c662b869d60: Sending DeleteTablet for 3 replicas of tablet 8961b35811b54a61be9ed425a2e304ef
I20260812 14:47:14.807837 17290 tablet_service.cc:1514] Processing DeleteTablet for tablet 8961b35811b54a61be9ed425a2e304ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:14 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56778
I20260812 14:47:14.807878 17562 tablet_service.cc:1514] Processing DeleteTablet for tablet 8961b35811b54a61be9ed425a2e304ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:14 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:34958
I20260812 14:47:14.807876 17822 tablet_service.cc:1514] Processing DeleteTablet for tablet 8961b35811b54a61be9ed425a2e304ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:14 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:58472
I20260812 14:47:14.808130 18287 tablet_replica.cc:330] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: stopping tablet replica
I20260812 14:47:14.808235 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17166
I20260812 14:47:14.808259 18287 raft_consensus.cc:2229] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:14.808316 18287 raft_consensus.cc:2258] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:14.813702 18287 ts_tablet_manager.cc:1905] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:14.815613 18287 ts_tablet_manager.cc:1918] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:47:14.817327 18287 log.cc:1196] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/8961b35811b54a61be9ed425a2e304ef
I20260812 14:47:14.817580 18287 ts_tablet_manager.cc:1939] T 8961b35811b54a61be9ed425a2e304ef P 59e34a2e80f1446ebf8651fd49167249: Deleting consensus metadata
I20260812 14:47:14.817991 18286 tablet_replica.cc:330] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: stopping tablet replica
I20260812 14:47:14.818055 17003 catalog_manager.cc:4841] TS 59e34a2e80f1446ebf8651fd49167249 (127.13.146.130:42293): tablet 8961b35811b54a61be9ed425a2e304ef (table test-table [id=72c471ffc0ce424ab22e35ac349bf804]) successfully deleted
I20260812 14:47:14.818140 18286 raft_consensus.cc:2229] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:14.818203 18286 raft_consensus.cc:2258] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:14.821887 18286 ts_tablet_manager.cc:1905] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:14.824120 18286 ts_tablet_manager.cc:1918] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:47:14.824206 18286 log.cc:1196] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/8961b35811b54a61be9ed425a2e304ef
I20260812 14:47:14.824445 18286 ts_tablet_manager.cc:1939] T 8961b35811b54a61be9ed425a2e304ef P 48725a06e6fd4e6abf7aa4cf1680b0cd: Deleting consensus metadata
I20260812 14:47:14.824885 17001 catalog_manager.cc:4841] TS 48725a06e6fd4e6abf7aa4cf1680b0cd (127.13.146.131:34335): tablet 8961b35811b54a61be9ed425a2e304ef (table test-table [id=72c471ffc0ce424ab22e35ac349bf804]) successfully deleted
W20260812 14:47:14.831512 17001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:40789 (error 108)
I20260812 14:47:14.831856 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17430
I20260812 14:47:14.838384 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:14.831984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:40789 (local address 127.0.0.1:43560)
0812 14:47:14.838246 (+ 6262us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:14.838325 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:40789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6222,"thread_start_us":68,"threads_started":1}
W20260812 14:47:14.841188 17001 catalog_manager.cc:4584] TS 2a6ec476acb6478987ca782cbc968b3c (127.13.146.129:40789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8961b35811b54a61be9ed425a2e304ef: Network error: Client connection negotiation failed: client connection to 127.13.146.129:40789: connect: Connection refused (error 111)
I20260812 14:47:14.853446 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17701
I20260812 14:47:14.874631 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16974
2026-08-12T14:47:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6264 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:14 dist-test-slave-8hhm krb5kdc[18296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:14 dist-test-slave-8hhm krb5kdc[18296](info): set up 2 sockets
Aug 12 14:47:14 dist-test-slave-8hhm krb5kdc[18296](info): commencing operation
krb5kdc: starting...
W20260812 14:47:16.209800 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:16.340188 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45895
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:16.424434 18322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:16.424705 18322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:16.424744 18322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:16.425948 18322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:16.426029 18322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:16.426052 18322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:16.426074 18322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:16.426105 18322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:16.427630 18322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:16.427920 18322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:16.428170 18322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:16.433471 18337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.433485 18334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.433471 18333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:16.433681 18322 server_base.cc:1034] running on GCE node
I20260812 14:47:16.434078 18322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:16.434419 18322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:16.435593 18322 hybrid_clock.cc:648] HybridClock initialized: now 1786546036435570 us; error 58 us; skew 500 ppm
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:16.442373 18322 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:16.442902 18322 webserver.cc:460] Webserver started at http://127.13.146.190:41141/ using document root <none> and password file <none>
I20260812 14:47:16.443143 18322 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:16.443221 18322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:16.443373 18322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:16.444501 18322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "222cc545ea484fa1997e16dddbd1ff70"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f4c382a1f034d44c3e42b362b7d0f620"
server_key_iv: "dbfd2688926f27d3dc6b252967850aad"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.444881 18322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "222cc545ea484fa1997e16dddbd1ff70"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f4c382a1f034d44c3e42b362b7d0f620"
server_key_iv: "dbfd2688926f27d3dc6b252967850aad"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.446354 18322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:16.447131 18353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.447347 18322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:16.447458 18322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "222cc545ea484fa1997e16dddbd1ff70"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f4c382a1f034d44c3e42b362b7d0f620"
server_key_iv: "dbfd2688926f27d3dc6b252967850aad"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.447535 18322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:16.459689 18322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:16.461833 18322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:16.462008 18322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:16.466672 18322 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:45895
I20260812 14:47:16.466722 18445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:45895 every 8 connection(s)
I20260812 14:47:16.467135 18322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:16.468058 18453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:16.470948 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70: Bootstrap starting.
I20260812 14:47:16.471566 18453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:16.471906 18453 log.cc:824] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:16.472683 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70: No bootstrap required, opened a new log
I20260812 14:47:16.474363 18453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } }
I20260812 14:47:16.474529 18453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:16.474577 18453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222cc545ea484fa1997e16dddbd1ff70, State: Initialized, Role: FOLLOWER
I20260812 14:47:16.474740 18453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } }
I20260812 14:47:16.474824 18453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:16.474869 18453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:16.474918 18453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:16.475435 18453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } }
I20260812 14:47:16.475557 18453 leader_election.cc:302] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 222cc545ea484fa1997e16dddbd1ff70; no voters:
I20260812 14:47:16.475798 18453 leader_election.cc:288] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:16.475826 18457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:16.475921 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18322
I20260812 14:47:16.475947 18457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [term 1 LEADER]: Becoming Leader. State: Replica: 222cc545ea484fa1997e16dddbd1ff70, State: Running, Role: LEADER
I20260812 14:47:16.476027 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:16.476032 18457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } }
I20260812 14:47:16.476097 18453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:16.476202 13898 external_mini_cluster.cc:1428] Setting key dee9a88bda1efe66146899489dfadc0a
I20260812 14:47:16.476519 18463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "222cc545ea484fa1997e16dddbd1ff70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } } }
I20260812 14:47:16.476612 18463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:16.476951 18465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 222cc545ea484fa1997e16dddbd1ff70. Latest consensus state: current_term: 1 leader_uuid: "222cc545ea484fa1997e16dddbd1ff70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222cc545ea484fa1997e16dddbd1ff70" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45895 } } }
I20260812 14:47:16.477102 18465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:16.477273 18473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:16.477878 18473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:16.479622 18473 catalog_manager.cc:1347] Generated new cluster ID: af485e4d20ff443284819254f840c66b
I20260812 14:47:16.479686 18473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:16.488607 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:16.476889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.13.146.190:45895)
0812 14:47:16.477567 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:16.477578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:16.477612 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:47:16.477742 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:16.477750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:16.477824 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:16.477958 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:16.478692 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.480066 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.481099 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.481765 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.481836 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:16.484829 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:16.484833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:16.484834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:16.484845 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:16.486921 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.487300 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.487303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.487303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.487399 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.487575 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.487575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.487575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.487851 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:16.487976 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:16.488235 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 14:47:16.488376 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":83,"threads_started":1}
I20260812 14:47:16.508395 18473 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:16.509019 18473 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:16.516963 18473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 222cc545ea484fa1997e16dddbd1ff70: Generated new TSK 0
I20260812 14:47:16.517174 18473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:16.547777 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:16.633952 18496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:16.634168 18496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:16.634192 18496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:16.635399 18496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:16.635445 18496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:16.635473 18496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:16.636947 18496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:16.637236 18496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:16.637491 18496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:16.643006 18505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.643006 18508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.643203 18504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:16.643173 18496 server_base.cc:1034] running on GCE node
I20260812 14:47:16.643460 18496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:16.643683 18496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:16.644831 18496 hybrid_clock.cc:648] HybridClock initialized: now 1786546036644818 us; error 30 us; skew 500 ppm
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:16.647537 18496 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:16.648036 18496 webserver.cc:460] Webserver started at http://127.13.146.129:39287/ using document root <none> and password file <none>
I20260812 14:47:16.648278 18496 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:16.648347 18496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:16.648504 18496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:16.649732 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "243491625f134ab2afc898e720e30640"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f0ae8b8d3a624724fbbb0765369543c7"
server_key_iv: "4925bb2ba9a1b6d5c885b7ee29d16c79"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.650112 18496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "243491625f134ab2afc898e720e30640"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f0ae8b8d3a624724fbbb0765369543c7"
server_key_iv: "4925bb2ba9a1b6d5c885b7ee29d16c79"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.651629 18496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:47:16.652468 18528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.652729 18496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:16.652856 18496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "243491625f134ab2afc898e720e30640"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "f0ae8b8d3a624724fbbb0765369543c7"
server_key_iv: "4925bb2ba9a1b6d5c885b7ee29d16c79"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.652940 18496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:16.665809 18496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:16.667834 18496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:16.668025 18496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:16.668294 18496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:16.668639 18496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:16.668696 18496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.668731 18496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:16.668759 18496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.674927 18496 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:41121
I20260812 14:47:16.674983 18733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:41121 every 8 connection(s)
I20260812 14:47:16.675331 18496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:16.683954 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18496
I20260812 14:47:16.684062 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:16.684265 13898 external_mini_cluster.cc:1428] Setting key da84a1a710486d0ed1912d4f1cbf69ed
I20260812 14:47:16.686551 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:16.676114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:54007 (local address 127.13.146.190:45895)
0812 14:47:16.676164 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:16.676167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:16.676401 (+ 234us) server_negotiation.cc:407] Connection header received
0812 14:47:16.677085 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:16.677087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:16.677098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:16.677175 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:16.678357 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.678861 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.679912 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.680182 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.680256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:16.682696 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:16.682697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:16.682698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:16.682706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:16.684637 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.685148 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.685149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.685150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.685270 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.685620 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.685621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.685622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.685884 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:16.685977 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:16.686398 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 14:47:16.686485 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:47:16.686902 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:16.676101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45895 (local address 127.13.146.129:54007)
0812 14:47:16.676289 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:16.676327 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:16.676982 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:16.677256 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:16.677259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:16.678032 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:16.678245 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:16.678252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.678954 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.678955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:16.679802 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:16.679805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.680365 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.680366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:16.680484 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:16.680987 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:16.680996 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:16.682582 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:16.684745 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:16.684753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:16.684756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:16.685065 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:16.685368 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:16.685369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:16.685369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:16.685548 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:16.686024 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:16.686032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:16.686226 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 14:47:16.686356 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 14:47:16.686569 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260812 14:47:16.687521 18734 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45895
I20260812 14:47:16.687652 18734 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:16.687853 18734 heartbeater.cc:511] Master 127.13.146.190:45895 requested a full tablet report, sending...
I20260812 14:47:16.688448 18387 ts_manager.cc:194] Registered new tserver with Master: 243491625f134ab2afc898e720e30640 (127.13.146.129:41121)
I20260812 14:47:16.689095 18387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:54007
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:16.711745 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:16.801167 18742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:16.801512 18742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:16.801561 18742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:16.803651 18742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:16.803720 18742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:16.803771 18742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:16.806118 18742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:16.806445 18742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:16.806708 18742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:16.812479 18749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.812548 18750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.812654 18752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:16.812870 18742 server_base.cc:1034] running on GCE node
I20260812 14:47:16.813119 18742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:16.813442 18742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:16.814602 18742 hybrid_clock.cc:648] HybridClock initialized: now 1786546036814573 us; error 45 us; skew 500 ppm
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:16.817327 18742 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:16.817778 18742 webserver.cc:460] Webserver started at http://127.13.146.130:43347/ using document root <none> and password file <none>
I20260812 14:47:16.818046 18742 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:16.818092 18742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:16.818254 18742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:16.819403 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "3a0a525a0a914382879485290ce60d40"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "42054180c8a95829fb7e87a7d8b09caf"
server_key_iv: "0f95cda086642927dd0c859c7c8260d3"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.819770 18742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "3a0a525a0a914382879485290ce60d40"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "42054180c8a95829fb7e87a7d8b09caf"
server_key_iv: "0f95cda086642927dd0c859c7c8260d3"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.821130 18742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:16.821949 18763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.822127 18742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:16.822227 18742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "3a0a525a0a914382879485290ce60d40"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "42054180c8a95829fb7e87a7d8b09caf"
server_key_iv: "0f95cda086642927dd0c859c7c8260d3"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.822305 18742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:16.839427 18742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:16.841456 18742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:16.841619 18742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:16.841871 18742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:16.842213 18742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:16.842272 18742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.842306 18742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:16.842335 18742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.848412 18742 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34959
I20260812 14:47:16.848470 18965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34959 every 8 connection(s)
I20260812 14:47:16.848799 18742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:16.857887 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18742
I20260812 14:47:16.858047 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:16.858302 13898 external_mini_cluster.cc:1428] Setting key 682f6baae2837203d154ad8df29ab685
I20260812 14:47:16.859071 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:16.849593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:55351 (local address 127.13.146.190:45895)
0812 14:47:16.849682 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:16.849684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:16.849900 (+ 216us) server_negotiation.cc:407] Connection header received
0812 14:47:16.850377 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:16.850378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:16.850389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:16.850454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:16.851426 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.851942 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.853043 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.853310 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.853381 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:16.855757 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:16.855758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:16.855759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:16.855766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:16.857452 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.857864 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.857865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.857865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.857935 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:16.858259 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:16.858260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:16.858260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:16.858415 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:16.858495 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:16.858855 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 14:47:16.858969 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 14:47:16.859306 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:16.849578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45895 (local address 127.13.146.130:55351)
0812 14:47:16.849761 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:16.849806 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:16.850295 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:16.850513 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:16.850516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:16.851127 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:16.851329 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:16.851334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.852061 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.852062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:16.852917 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:16.852919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:16.853412 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:16.853414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:16.853542 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:16.854082 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:16.854090 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:16.855662 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:16.857529 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:16.857537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:16.857539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:16.857780 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:16.858030 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:16.858031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:16.858032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:16.858188 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:16.858510 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:16.858519 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:16.858712 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 14:47:16.858825 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 14:47:16.859000 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260812 14:47:16.859829 18966 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45895
I20260812 14:47:16.859918 18966 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:16.860256 18966 heartbeater.cc:511] Master 127.13.146.190:45895 requested a full tablet report, sending...
I20260812 14:47:16.860672 18387 ts_manager.cc:194] Registered new tserver with Master: 3a0a525a0a914382879485290ce60d40 (127.13.146.130:34959)
I20260812 14:47:16.861166 18387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:55351
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:16.888284 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:16.977810 18974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:16.978051 18974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:16.978080 18974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:16.979338 18974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:16.979386 18974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:16.979413 18974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:16.980975 18974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:16.981359 18974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:16.981611 18974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:16.986884 18988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.986900 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:16.986857 18986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:16.987284 18974 server_base.cc:1034] running on GCE node
I20260812 14:47:16.987540 18974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:16.987782 18974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:16.988932 18974 hybrid_clock.cc:648] HybridClock initialized: now 1786546036988917 us; error 26 us; skew 500 ppm
Aug 12 14:47:16 dist-test-slave-8hhm krb5kdc[18296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:16.991808 18974 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:16.992350 18974 webserver.cc:460] Webserver started at http://127.13.146.131:36231/ using document root <none> and password file <none>
I20260812 14:47:16.992610 18974 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:16.992687 18974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:16.992851 18974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:16.994042 18974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "8bf7a823fc5f474babef0a672b376b55"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "9b39790293d593bda793090cdf0cdb47"
server_key_iv: "d91233cfb238272d5564853a995fdabb"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.994421 18974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "8bf7a823fc5f474babef0a672b376b55"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "9b39790293d593bda793090cdf0cdb47"
server_key_iv: "d91233cfb238272d5564853a995fdabb"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.995808 18974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:16.996598 19004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.996800 18974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:16.996913 18974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "8bf7a823fc5f474babef0a672b376b55"
format_stamp: "Formatted at 2026-08-12 14:47:16 on dist-test-slave-8hhm"
server_key: "9b39790293d593bda793090cdf0cdb47"
server_key_iv: "d91233cfb238272d5564853a995fdabb"
server_key_version: "encryptionkey@0"
I20260812 14:47:16.996991 18974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:17.019054 18974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:17.021638 18974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:17.021811 18974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:17.022081 18974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:17.022410 18974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:17.022465 18974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:17.022512 18974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:17.022534 18974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:17.028587 18974 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36593
I20260812 14:47:17.028631 19218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36593 every 8 connection(s)
I20260812 14:47:17.028960 18974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:17.035161 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18974
I20260812 14:47:17.035277 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
Aug 12 14:47:17 dist-test-slave-8hhm krb5kdc[18296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546036, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:17.035532 13898 external_mini_cluster.cc:1428] Setting key b1135328b9ffb9978db92326f526f16d
I20260812 14:47:17.038873 18476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:17.029761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47995 (local address 127.13.146.190:45895)
0812 14:47:17.029818 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:17.029820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:17.030031 (+ 211us) server_negotiation.cc:407] Connection header received
0812 14:47:17.030571 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:17.030573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:17.030585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:17.030664 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:17.031584 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:17.032136 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:17.033095 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:17.033378 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:17.033439 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:17.035939 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:17.035941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:17.035942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:17.035951 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:17.037622 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:17.037993 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:17.037993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:17.037994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:17.038050 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:17.038272 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:17.038273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:17.038273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:17.038384 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:17.038440 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:17.038712 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 14:47:17.038802 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:47:17.038935 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:17.029704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45895 (local address 127.13.146.131:47995)
0812 14:47:17.029869 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:17.029913 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:17.030468 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:17.030720 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:17.030724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:17.031301 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:17.031496 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:17.031500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:17.032225 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:17.032226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:17.033007 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:17.033009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:17.033480 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:17.033481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:17.033595 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:17.034197 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:17.034206 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:17.035830 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:17.037697 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:17.037703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:17.037704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:17.037923 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:17.038110 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:17.038111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:17.038111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:17.038213 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:17.038442 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:17.038447 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:17.038559 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 14:47:17.038652 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 14:47:17.038772 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260812 14:47:17.039459 19219 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45895
I20260812 14:47:17.039592 19219 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:17.039789 19219 heartbeater.cc:511] Master 127.13.146.190:45895 requested a full tablet report, sending...
I20260812 14:47:17.040177 18387 ts_manager.cc:194] Registered new tserver with Master: 8bf7a823fc5f474babef0a672b376b55 (127.13.146.131:36593)
I20260812 14:47:17.040517 18387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:47995
I20260812 14:47:17.047612 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:47:17.061077 18476 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:17.059958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.13.146.190:45895)
0812 14:47:17.060056 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:17.060059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:17.060166 (+ 107us) server_negotiation.cc:407] Connection header received
0812 14:47:17.060273 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:17.060274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:17.060285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:17.060359 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:17.060990 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50132: BlockingRecv error: recv got EOF from 127.0.0.1:50132 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 14:47:17.061928 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18496
I20260812 14:47:17.076781 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18742
I20260812 14:47:17.090646 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18974
I20260812 14:47:17.106050 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18322
2026-08-12T14:47:17Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2222 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:17 dist-test-slave-8hhm krb5kdc[19239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:17 dist-test-slave-8hhm krb5kdc[19239](info): set up 2 sockets
Aug 12 14:47:17 dist-test-slave-8hhm krb5kdc[19239](info): commencing operation
krb5kdc: starting...
W20260812 14:47:18.410526 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:18.540566 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:43409
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:18.626518 19261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:18.626791 19261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:18.626834 19261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:18.628053 19261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:18.628137 19261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:18.628165 19261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:18.628187 19261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:18.628223 19261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:18.629908 19261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:43409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:43409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:18.630229 19261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:18.630493 19261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:18.636165 19279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:18.636147 19276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:18.636145 19277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:18.636215 19261 server_base.cc:1034] running on GCE node
I20260812 14:47:18.636587 19261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:18.636905 19261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:18.638089 19261 hybrid_clock.cc:648] HybridClock initialized: now 1786546038638062 us; error 43 us; skew 500 ppm
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546038, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:18.641065 19261 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:18.641688 19261 webserver.cc:460] Webserver started at http://127.13.146.190:35223/ using document root <none> and password file <none>
I20260812 14:47:18.641933 19261 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:18.642001 19261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:18.642154 19261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:18.643321 19261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "3ed29928818044838352fbb44006a298"
format_stamp: "Formatted at 2026-08-12 14:47:18 on dist-test-slave-8hhm"
server_key: "e8c62974352fbee605b123f4f86c25cb"
server_key_iv: "9e2ffb9a632666ea02baa7ca06da8737"
server_key_version: "encryptionkey@0"
I20260812 14:47:18.643743 19261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "3ed29928818044838352fbb44006a298"
format_stamp: "Formatted at 2026-08-12 14:47:18 on dist-test-slave-8hhm"
server_key: "e8c62974352fbee605b123f4f86c25cb"
server_key_iv: "9e2ffb9a632666ea02baa7ca06da8737"
server_key_version: "encryptionkey@0"
I20260812 14:47:18.645196 19261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:18.646207 19296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:18.646641 19261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:18.646762 19261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "3ed29928818044838352fbb44006a298"
format_stamp: "Formatted at 2026-08-12 14:47:18 on dist-test-slave-8hhm"
server_key: "e8c62974352fbee605b123f4f86c25cb"
server_key_iv: "9e2ffb9a632666ea02baa7ca06da8737"
server_key_version: "encryptionkey@0"
I20260812 14:47:18.646842 19261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:18.655916 19261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:18.658378 19261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:18.658560 19261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:18.663048 19261 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:43409
I20260812 14:47:18.663100 19404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:43409 every 8 connection(s)
I20260812 14:47:18.663695 19261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:18.664394 19405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:18.666301 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19261
I20260812 14:47:18.666381 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:18.666594 13898 external_mini_cluster.cc:1428] Setting key c2ec035e1f0594cc2f9b09ded2460fe1
I20260812 14:47:18.667697 19405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298: Bootstrap starting.
I20260812 14:47:18.668478 19405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:18.668845 19405 log.cc:824] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:18.669675 19405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298: No bootstrap required, opened a new log
I20260812 14:47:18.672103 19405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } }
I20260812 14:47:18.672305 19405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:18.672348 19405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed29928818044838352fbb44006a298, State: Initialized, Role: FOLLOWER
I20260812 14:47:18.672480 19405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } }
I20260812 14:47:18.672585 19405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:18.672633 19405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:18.672724 19405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:18.673604 19405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } }
I20260812 14:47:18.673699 19405 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed29928818044838352fbb44006a298; no voters:
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:18.673959 19405 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:18.673980 19416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:18.674182 19416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [term 1 LEADER]: Becoming Leader. State: Replica: 3ed29928818044838352fbb44006a298, State: Running, Role: LEADER
I20260812 14:47:18.674302 19405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:18.674335 19416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } }
I20260812 14:47:18.674749 19416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed29928818044838352fbb44006a298. Latest consensus state: current_term: 1 leader_uuid: "3ed29928818044838352fbb44006a298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } } }
I20260812 14:47:18.675016 19416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:18.674933 19417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed29928818044838352fbb44006a298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed29928818044838352fbb44006a298" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43409 } } }
I20260812 14:47:18.675297 19417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:18.675475 19435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:18.676371 19435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:18.678087 19435 catalog_manager.cc:1347] Generated new cluster ID: cd94f1cdda7a4c5e989f1c23dce8f23a
I20260812 14:47:18.678141 19435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:18.678717 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:18.667273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.13.146.190:43409)
0812 14:47:18.667471 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:18.667479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:18.667509 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:47:18.667962 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:18.667970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:18.668020 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:18.668152 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:18.669059 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:18.670059 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:18.671081 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:18.671475 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:18.671576 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:18.674586 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:18.674591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:18.674592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:18.674604 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:18.676802 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:18.677255 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:18.677259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:18.677259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:18.677367 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:18.677620 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:18.677621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:18.677621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:18.677946 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:18.678099 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:18.678376 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 14:47:18.678515 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260812 14:47:18.679719 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:47:18.691449 19435 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:18.692363 19435 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:18.695523 19435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed29928818044838352fbb44006a298: Generated new TSK 0
I20260812 14:47:18.695722 19435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:18.707273 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:18.702511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.13.146.190:43409)
0812 14:47:18.702561 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:18.702563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:18.702707 (+ 144us) server_negotiation.cc:407] Connection header received
0812 14:47:18.702785 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:18.702786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:18.702803 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:18.702885 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:18.703504 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:18.704346 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:18.705585 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:18.705818 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:18.705910 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:18.706736 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:18.706738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:18.706738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:18.706950 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:18.706969 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:18.706975 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:18.707043 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:18.707123 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:47:18.707167 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:47:18.710636 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19261
2026-08-12T14:47:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1611 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19462](info): set up 2 sockets
Aug 12 14:47:18 dist-test-slave-8hhm krb5kdc[19462](info): commencing operation
krb5kdc: starting...
W20260812 14:47:20.032999 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:20.157779 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46809
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:20.243366 19484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:20.243646 19484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:20.243690 19484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:20.244853 19484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:20.244930 19484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:20.244952 19484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:20.244971 19484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:20.245003 19484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:20.246544 19484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:20.246842 19484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:20.247085 19484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:20.252359 19495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.252326 19491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.252333 19493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:20.252683 19484 server_base.cc:1034] running on GCE node
I20260812 14:47:20.252964 19484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:20.253333 19484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:20.254546 19484 hybrid_clock.cc:648] HybridClock initialized: now 1786546040254521 us; error 39 us; skew 500 ppm
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:20.257288 19484 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:20.257726 19484 webserver.cc:460] Webserver started at http://127.13.146.190:37879/ using document root <none> and password file <none>
I20260812 14:47:20.257949 19484 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:20.258004 19484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:20.258100 19484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:20.259189 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "f8a9138a617d44b9abae2baa477753a1"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "a035286525b87ab3f21af4cd6e0c40fc"
server_key_iv: "c39e748faa7fa5948e3eda98899196c8"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.259511 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "f8a9138a617d44b9abae2baa477753a1"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "a035286525b87ab3f21af4cd6e0c40fc"
server_key_iv: "c39e748faa7fa5948e3eda98899196c8"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.260910 19484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:20.261809 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.262006 19484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:20.262106 19484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "f8a9138a617d44b9abae2baa477753a1"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "a035286525b87ab3f21af4cd6e0c40fc"
server_key_iv: "c39e748faa7fa5948e3eda98899196c8"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.262185 19484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:20.279367 19484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:20.281445 19484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:20.281566 19484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:20.285718 19484 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:46809
I20260812 14:47:20.285776 19603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:46809 every 8 connection(s)
I20260812 14:47:20.286154 19484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:20.287336 19604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:20.289825 19604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: Bootstrap starting.
I20260812 14:47:20.290438 19604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:20.290764 19604 log.cc:824] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:20.291517 19604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: No bootstrap required, opened a new log
I20260812 14:47:20.293378 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19484
I20260812 14:47:20.293198 19604 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } }
I20260812 14:47:20.293443 19604 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:20.293489 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:20.293488 19604 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8a9138a617d44b9abae2baa477753a1, State: Initialized, Role: FOLLOWER
I20260812 14:47:20.293648 19604 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } }
I20260812 14:47:20.293697 13898 external_mini_cluster.cc:1428] Setting key 8a1f024f0f925099d830dee744266ad6
I20260812 14:47:20.293737 19604 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:20.293785 19604 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:20.293839 19604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:20.294960 19604 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } }
I20260812 14:47:20.295084 19604 leader_election.cc:302] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8a9138a617d44b9abae2baa477753a1; no voters:
I20260812 14:47:20.295269 19604 leader_election.cc:288] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:20.295300 19611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:20.295467 19611 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [term 1 LEADER]: Becoming Leader. State: Replica: f8a9138a617d44b9abae2baa477753a1, State: Running, Role: LEADER
I20260812 14:47:20.295548 19604 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:20.295742 19611 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } }
I20260812 14:47:20.296746 19612 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8a9138a617d44b9abae2baa477753a1. Latest consensus state: current_term: 1 leader_uuid: "f8a9138a617d44b9abae2baa477753a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } } }
I20260812 14:47:20.296845 19612 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:20.297142 19613 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8a9138a617d44b9abae2baa477753a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a9138a617d44b9abae2baa477753a1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46809 } } }
I20260812 14:47:20.297401 19613 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1 [sys.catalog]: This master's current role is: LEADER
W20260812 14:47:20.297593 19638 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:20.297658 19638 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:20.297708 19641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:20.297871 19641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:20.299470 19641 catalog_manager.cc:1347] Generated new cluster ID: b695cc29d8eb49eb8ba675b59bb5206d
I20260812 14:47:20.299521 19641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:20.306845 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.294248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.13.146.190:46809)
0812 14:47:20.294459 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:20.294468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:20.294620 (+ 152us) server_negotiation.cc:407] Connection header received
0812 14:47:20.294773 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:20.294778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:20.294822 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:20.294935 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:20.295831 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.297468 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.298542 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.299166 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.299301 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.301978 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:20.301982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:20.301983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:20.301995 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:20.304788 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.305188 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.305192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.305192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.305352 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.305539 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.305540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.305540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.305837 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:20.305975 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:20.306470 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 14:47:20.306608 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260812 14:47:20.316815 19641 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:20.317472 19641 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:20.325289 19641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8a9138a617d44b9abae2baa477753a1: Generated new TSK 0
I20260812 14:47:20.325523 19641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:20.358062 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:20.451929 19651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:20.452136 19651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:20.452159 19651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:20.453305 19651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:20.453346 19651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:20.453372 19651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:20.454807 19651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:20.454998 19651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:20.455174 19651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:20.460561 19663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.460561 19659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.460600 19660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:20.460644 19651 server_base.cc:1034] running on GCE node
I20260812 14:47:20.461063 19651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:20.461423 19651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:20.462598 19651 hybrid_clock.cc:648] HybridClock initialized: now 1786546040462591 us; error 40 us; skew 500 ppm
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:20.465624 19651 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:20.468478 19651 webserver.cc:460] Webserver started at http://127.13.146.129:38865/ using document root <none> and password file <none>
I20260812 14:47:20.468752 19651 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:20.468797 19651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:20.468894 19651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:20.470073 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "b99418a326264d878bb2c26a05a1b59f"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "64af759ecc400ec3569136a2be1e0ede"
server_key_iv: "89529e49c2f9f684f41e533a01daeffa"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.470505 19651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "b99418a326264d878bb2c26a05a1b59f"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "64af759ecc400ec3569136a2be1e0ede"
server_key_iv: "89529e49c2f9f684f41e533a01daeffa"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.471894 19651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:20.472643 19672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.472829 19651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:20.472937 19651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "b99418a326264d878bb2c26a05a1b59f"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "64af759ecc400ec3569136a2be1e0ede"
server_key_iv: "89529e49c2f9f684f41e533a01daeffa"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.473028 19651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:20.485709 19651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:20.488277 19651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:20.488456 19651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:20.488713 19651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:20.489054 19651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:20.489130 19651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.489204 19651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:20.489267 19651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.495663 19651 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:42317
I20260812 14:47:20.495739 19870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:42317 every 8 connection(s)
I20260812 14:47:20.496168 19651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:20.504704 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19651
I20260812 14:47:20.504834 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:20.505039 13898 external_mini_cluster.cc:1428] Setting key 4e855fb4e66a24e97cbb1c88943424f4
I20260812 14:47:20.506312 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.497076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:58541 (local address 127.13.146.190:46809)
0812 14:47:20.497141 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:20.497143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:20.497546 (+ 403us) server_negotiation.cc:407] Connection header received
0812 14:47:20.498250 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:20.498251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:20.498267 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:20.498321 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:20.499301 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.499828 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.500858 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.501069 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.501149 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.503348 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:20.503349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:20.503350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:20.503357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:20.504582 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.505105 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.505107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.505107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.505193 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.505505 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.505506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.505506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.505659 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:20.505728 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:20.506069 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 14:47:20.506180 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:47:20.506337 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.497075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46809 (local address 127.13.146.129:58541)
0812 14:47:20.497403 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:20.497443 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:20.498138 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:20.498391 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:20.498395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:20.498988 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:20.499192 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.499197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.499938 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.499939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.500762 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.500764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.501159 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.501161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.501309 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.501827 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:20.501835 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:20.503243 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:20.504711 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.504719 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.504721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.505026 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.505305 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.505306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.505307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.505444 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.505743 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:20.505752 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:20.505918 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 14:47:20.506017 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 14:47:20.506170 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":143,"threads_started":1}
I20260812 14:47:20.507028 19872 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46809
I20260812 14:47:20.507164 19872 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:20.507356 19872 heartbeater.cc:511] Master 127.13.146.190:46809 requested a full tablet report, sending...
I20260812 14:47:20.507929 19534 ts_manager.cc:194] Registered new tserver with Master: b99418a326264d878bb2c26a05a1b59f (127.13.146.129:42317)
I20260812 14:47:20.508565 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:58541
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:20.531589 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:20.616199 19887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:20.616514 19887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:20.616557 19887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:20.617805 19887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:20.617869 19887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:20.617913 19887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:20.619381 19887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:20.619660 19887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:20.619899 19887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:20.625192 19899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:20.625316 19887 server_base.cc:1034] running on GCE node
W20260812 14:47:20.625202 19898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.625206 19902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:20.625702 19887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:20.625932 19887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:20.627130 19887 hybrid_clock.cc:648] HybridClock initialized: now 1786546040627111 us; error 27 us; skew 500 ppm
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:20.629840 19887 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:20.630316 19887 webserver.cc:460] Webserver started at http://127.13.146.130:33359/ using document root <none> and password file <none>
I20260812 14:47:20.630511 19887 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:20.630553 19887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:20.630649 19887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:20.631699 19887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "dbb9dc2b84b24ae1aaab9c8bab2d5c76"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "10e224ad967712009d5c73343683df7d"
server_key_iv: "9d20a13738fb3c21b7c48215eb04adc7"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.632038 19887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "dbb9dc2b84b24ae1aaab9c8bab2d5c76"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "10e224ad967712009d5c73343683df7d"
server_key_iv: "9d20a13738fb3c21b7c48215eb04adc7"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.633452 19887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:20.634178 19917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.634370 19887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:20.634469 19887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "dbb9dc2b84b24ae1aaab9c8bab2d5c76"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "10e224ad967712009d5c73343683df7d"
server_key_iv: "9d20a13738fb3c21b7c48215eb04adc7"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.634531 19887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:20.656265 19887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:20.658810 19887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:20.658931 19887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:20.659160 19887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:20.659448 19887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:20.659482 19887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.659504 19887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:20.659516 19887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.665337 19887 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:44723
I20260812 14:47:20.665441 20123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:44723 every 8 connection(s)
I20260812 14:47:20.665757 19887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:20.667220 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19887
I20260812 14:47:20.667358 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:20.667573 13898 external_mini_cluster.cc:1428] Setting key 3ac80e87bc5d382ab776591e1ca9f557
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:20.675825 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.666526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:47731 (local address 127.13.146.190:46809)
0812 14:47:20.666591 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:20.666593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:20.666786 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:47:20.667327 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:20.667329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:20.667346 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:20.667420 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:20.668620 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.669403 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.670456 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.670675 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.670753 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.672959 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:20.672961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:20.672962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:20.672970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:20.674412 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.674830 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.674832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.674833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.674894 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.675158 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.675159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.675159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.675279 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:20.675354 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:20.675636 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 14:47:20.675724 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:47:20.676070 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.666508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46809 (local address 127.13.146.130:47731)
0812 14:47:20.666671 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:20.666708 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:20.667216 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:20.667518 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:20.667521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:20.668288 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:20.668525 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.668531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.669504 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.669505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.670349 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.670351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.670760 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.670761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.670890 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.671444 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:20.671452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:20.672858 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:20.674506 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.674514 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.674516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.674751 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.674971 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.674972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.674973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.675084 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.675357 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:20.675365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:20.675503 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 14:47:20.675614 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 14:47:20.675805 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260812 14:47:20.676499 20125 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46809
I20260812 14:47:20.676594 20125 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:20.676818 20125 heartbeater.cc:511] Master 127.13.146.190:46809 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:47:20.677313 19534 ts_manager.cc:194] Registered new tserver with Master: dbb9dc2b84b24ae1aaab9c8bab2d5c76 (127.13.146.130:44723)
I20260812 14:47:20.677696 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:47731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:20.695987 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:20.781570 20135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:20.781796 20135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:20.781821 20135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:20.783063 20135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:20.783110 20135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:20.783138 20135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:20.784622 20135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:20.784881 20135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:20.785156 20135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:20.790356 20146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.790452 20147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:20.790369 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:20.790591 20135 server_base.cc:1034] running on GCE node
I20260812 14:47:20.790834 20135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:20.791112 20135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:20.792303 20135 hybrid_clock.cc:648] HybridClock initialized: now 1786546040792277 us; error 39 us; skew 500 ppm
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:20.795104 20135 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:20.795629 20135 webserver.cc:460] Webserver started at http://127.13.146.131:32893/ using document root <none> and password file <none>
I20260812 14:47:20.795884 20135 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:20.795953 20135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:20.796108 20135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:20.797452 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "3ec32f90f3db4a2bad3a34c471f9df13"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "6cc58f0c833d03ba24fa9a653aa3128c"
server_key_iv: "02894fc0b0284830100e4ea30a0a5a04"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.797837 20135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "3ec32f90f3db4a2bad3a34c471f9df13"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "6cc58f0c833d03ba24fa9a653aa3128c"
server_key_iv: "02894fc0b0284830100e4ea30a0a5a04"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.799211 20135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:20.799943 20168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.800143 20135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:20.800246 20135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "3ec32f90f3db4a2bad3a34c471f9df13"
format_stamp: "Formatted at 2026-08-12 14:47:20 on dist-test-slave-8hhm"
server_key: "6cc58f0c833d03ba24fa9a653aa3128c"
server_key_iv: "02894fc0b0284830100e4ea30a0a5a04"
server_key_version: "encryptionkey@0"
I20260812 14:47:20.800330 20135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:20.813537 20135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:20.816032 20135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:20.816231 20135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:20.816483 20135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:20.816823 20135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:20.816879 20135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.816917 20135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:20.816965 20135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:20.823148 20135 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:42227
I20260812 14:47:20.823190 20368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:42227 every 8 connection(s)
I20260812 14:47:20.823559 20135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:20.832248 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20135
I20260812 14:47:20.832369 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:20.832571 13898 external_mini_cluster.cc:1428] Setting key 46efa526a91729900ed0b04f108938a6
I20260812 14:47:20.833015 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.824414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51543 (local address 127.13.146.190:46809)
0812 14:47:20.824463 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:20.824470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:20.824657 (+ 187us) server_negotiation.cc:407] Connection header received
0812 14:47:20.825157 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:20.825158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:20.825186 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:20.825302 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:20.826352 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.826869 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.827874 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.828068 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.828138 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.830344 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:20.830346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:20.830346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:20.830352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:20.831611 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.831985 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.831985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.831986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.832063 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.832299 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.832300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.832300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.832452 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:20.832528 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:20.832839 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 14:47:20.832925 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:47:20.833206 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.824355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46809 (local address 127.13.146.131:51543)
0812 14:47:20.824520 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:20.824564 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:20.825051 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:20.825392 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:20.825396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:20.826025 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:20.826251 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.826256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.826957 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.826957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.827778 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:20.827780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.828153 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.828154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:20.828257 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.828820 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:20.828828 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:20.830258 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:20.831686 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.831692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.831694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.831920 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.832131 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:20.832131 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:20.832132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:20.832232 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:20.832525 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:20.832530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:20.832686 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 14:47:20.832796 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:47:20.832958 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260812 14:47:20.833772 20370 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46809
I20260812 14:47:20.833869 20370 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:20.834014 20370 heartbeater.cc:511] Master 127.13.146.190:46809 requested a full tablet report, sending...
I20260812 14:47:20.834367 19534 ts_manager.cc:194] Registered new tserver with Master: 3ec32f90f3db4a2bad3a34c471f9df13 (127.13.146.131:42227)
I20260812 14:47:20.834692 19534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:51543
I20260812 14:47:20.844642 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[19462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546040, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:20.870302 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:20.862664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.13.146.190:46809)
0812 14:47:20.862768 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:20.862771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:20.862815 (+ 44us) server_negotiation.cc:407] Connection header received
0812 14:47:20.862883 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:20.862884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:20.862894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:20.862947 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:20.863522 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.864009 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.864874 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:20.865107 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:20.865171 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:20.867511 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:20.867512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:20.867513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:20.867519 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:20.869349 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.869647 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.869648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.869648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.869702 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:20.869837 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:20.869837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:20.869837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:20.869982 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:20.870087 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:20.870146 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:47:20.870210 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 14:47:20.870612 19534 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58160
I20260812 14:47:20.871083 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19651
I20260812 14:47:20.890100 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19887
I20260812 14:47:20.899582 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20135
I20260812 14:47:20.925632 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19484
2026-08-12T14:47:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2212 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[20400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[20400](info): set up 2 sockets
Aug 12 14:47:20 dist-test-slave-8hhm krb5kdc[20400](info): commencing operation
krb5kdc: starting...
W20260812 14:47:22.241990 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:22.368468 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34973
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:22.453050 20421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:22.453306 20421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:22.453330 20421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:22.454533 20421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:22.454583 20421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:22.454597 20421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:22.454649 20421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:22.454681 20421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:22.456243 20421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:22.456528 20421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:22.456780 20421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:22.461972 20427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:22.461938 20428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:22.461912 20434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:22.462080 20421 server_base.cc:1034] running on GCE node
I20260812 14:47:22.462533 20421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:22.462872 20421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:22.464047 20421 hybrid_clock.cc:648] HybridClock initialized: now 1786546042464031 us; error 47 us; skew 500 ppm
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:22.471307 20421 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:22.471884 20421 webserver.cc:460] Webserver started at http://127.13.146.190:40285/ using document root <none> and password file <none>
I20260812 14:47:22.472146 20421 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:22.472219 20421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:22.472393 20421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:22.473615 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "f11ee6c5c4264374bacce3a802328c4b"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "2ed04a378d9b356905cc3312c32d6fb1"
server_key_iv: "4b47dee2a1c6fb2084a465467b4677b4"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.474062 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "f11ee6c5c4264374bacce3a802328c4b"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "2ed04a378d9b356905cc3312c32d6fb1"
server_key_iv: "4b47dee2a1c6fb2084a465467b4677b4"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.475461 20421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:22.476238 20442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.476420 20421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:22.476536 20421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "f11ee6c5c4264374bacce3a802328c4b"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "2ed04a378d9b356905cc3312c32d6fb1"
server_key_iv: "4b47dee2a1c6fb2084a465467b4677b4"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.476625 20421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:22.490334 20421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:22.492833 20421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:22.493008 20421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:22.497407 20421 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34973
I20260812 14:47:22.497458 20544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34973 every 8 connection(s)
I20260812 14:47:22.498011 20421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:22.498735 20545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:22.501164 20545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b: Bootstrap starting.
I20260812 14:47:22.501832 20545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:22.502163 20545 log.cc:824] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:22.502871 20545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b: No bootstrap required, opened a new log
I20260812 14:47:22.504278 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20421
I20260812 14:47:22.504413 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:22.504596 13898 external_mini_cluster.cc:1428] Setting key 04fa601da7b11f432fe61938e907459b
I20260812 14:47:22.504511 20545 raft_consensus.cc:348] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } }
I20260812 14:47:22.504684 20545 raft_consensus.cc:374] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:22.504730 20545 raft_consensus.cc:729] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11ee6c5c4264374bacce3a802328c4b, State: Initialized, Role: FOLLOWER
I20260812 14:47:22.504899 20545 consensus_queue.cc:260] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } }
I20260812 14:47:22.504977 20545 raft_consensus.cc:388] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:22.505019 20545 raft_consensus.cc:482] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:22.505065 20545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:22.505834 20545 raft_consensus.cc:504] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } }
I20260812 14:47:22.505937 20545 leader_election.cc:302] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f11ee6c5c4264374bacce3a802328c4b; no voters:
I20260812 14:47:22.506117 20545 leader_election.cc:288] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:22.506158 20555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:22.506408 20555 raft_consensus.cc:686] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [term 1 LEADER]: Becoming Leader. State: Replica: f11ee6c5c4264374bacce3a802328c4b, State: Running, Role: LEADER
I20260812 14:47:22.506500 20545 sys_catalog.cc:564] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:22.506539 20555 consensus_queue.cc:237] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } }
I20260812 14:47:22.507002 20558 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f11ee6c5c4264374bacce3a802328c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } } }
I20260812 14:47:22.507088 20558 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:22.507315 20559 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f11ee6c5c4264374bacce3a802328c4b. Latest consensus state: current_term: 1 leader_uuid: "f11ee6c5c4264374bacce3a802328c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11ee6c5c4264374bacce3a802328c4b" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34973 } } }
I20260812 14:47:22.507640 20559 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:22.507812 20566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:22.508353 20566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:22.510028 20566 catalog_manager.cc:1347] Generated new cluster ID: b83892b268f94a1a8060133ffc004fc8
I20260812 14:47:22.510082 20566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:22.516441 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:22.505095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.13.146.190:34973)
0812 14:47:22.505776 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:22.505784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:22.505810 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:47:22.505941 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:22.505948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:22.506023 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:22.506179 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:22.507085 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.508589 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.509584 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.510190 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.510315 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:22.512647 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:22.512651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:22.512651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:22.512663 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:22.514850 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.515258 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.515261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.515262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.515362 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.515575 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.515576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.515576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.515820 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:22.516022 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:22.516091 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 14:47:22.516232 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":591,"threads_started":1}
I20260812 14:47:22.542111 20566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:22.542711 20566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:22.547196 20566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f11ee6c5c4264374bacce3a802328c4b: Generated new TSK 0
I20260812 14:47:22.547405 20566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:22.581081 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:22.678189 20587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:22.678390 20587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:22.678412 20587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:22.679638 20587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:22.679682 20587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:22.679709 20587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:22.681313 20587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:22.681499 20587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:22.681675 20587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:47:22.686939 20587 server_base.cc:1034] running on GCE node
W20260812 14:47:22.686980 20601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:22.687031 20597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:22.687007 20595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:22.687493 20587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:22.687796 20587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:22.688994 20587 hybrid_clock.cc:648] HybridClock initialized: now 1786546042688968 us; error 42 us; skew 500 ppm
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:22.691815 20587 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:22.692278 20587 webserver.cc:460] Webserver started at http://127.13.146.129:46081/ using document root <none> and password file <none>
I20260812 14:47:22.692492 20587 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:22.692538 20587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:22.692631 20587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:22.693761 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "1078664f7cef412f89a3dc4e4eb1bf9f"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "9319e16808504f1819f541d883f2e6ee"
server_key_iv: "81d6ac10cbfd4bea7283e865d7e692d7"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.694088 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "1078664f7cef412f89a3dc4e4eb1bf9f"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "9319e16808504f1819f541d883f2e6ee"
server_key_iv: "81d6ac10cbfd4bea7283e865d7e692d7"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.695380 20587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:22.696098 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.696328 20587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:22.696414 20587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "1078664f7cef412f89a3dc4e4eb1bf9f"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "9319e16808504f1819f541d883f2e6ee"
server_key_iv: "81d6ac10cbfd4bea7283e865d7e692d7"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.696471 20587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:22.706396 20587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:22.708791 20587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:22.708907 20587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:22.709100 20587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:22.709471 20587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:22.709507 20587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.709529 20587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:22.709542 20587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.715379 20587 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:39231
I20260812 14:47:22.715456 20830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:39231 every 8 connection(s)
I20260812 14:47:22.715788 20587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:22.717819 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20587
I20260812 14:47:22.717947 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:22.718154 13898 external_mini_cluster.cc:1428] Setting key b933cb42227a653233df6bf2a9d8ccc4
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:22.727020 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:22.716573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:49179 (local address 127.13.146.190:34973)
0812 14:47:22.716630 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:22.716632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:22.716830 (+ 198us) server_negotiation.cc:407] Connection header received
0812 14:47:22.718086 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:22.718088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:22.718103 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:22.718168 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:22.719032 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.719509 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.720999 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.721373 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.721469 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:22.723678 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:22.723679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:22.723680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:22.723691 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:22.725517 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.725945 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.725946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.725947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.726026 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.726279 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.726280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.726280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.726441 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:22.726521 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:22.726791 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 14:47:22.726892 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:22.727043 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:22.716524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34973 (local address 127.13.146.129:49179)
0812 14:47:22.716689 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:22.716734 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:22.717908 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:22.718252 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:22.718256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:22.718635 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:22.718896 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:22.718902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.719612 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.719613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:22.720899 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:22.720902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.721477 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.721478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:22.721585 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:22.722154 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:22.722162 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:22.723568 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:22.725624 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:22.725632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:22.725633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:22.725866 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:22.726101 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:22.726102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:22.726102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:22.726208 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:22.726514 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:22.726519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:22.726650 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 14:47:22.726736 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:47:22.726877 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260812 14:47:22.727689 20833 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34973
I20260812 14:47:22.727810 20833 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:22.728031 20833 heartbeater.cc:511] Master 127.13.146.190:34973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
I20260812 14:47:22.728618 20478 ts_manager.cc:194] Registered new tserver with Master: 1078664f7cef412f89a3dc4e4eb1bf9f (127.13.146.129:39231)
I20260812 14:47:22.729301 20478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:49179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:22.748034 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:22.832763 20848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:22.832973 20848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:22.832996 20848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:22.834233 20848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:22.834280 20848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:22.834306 20848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:22.835755 20848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:22.835973 20848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:22.836254 20848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:22.841711 20854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:22.841675 20865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:22.841732 20848 server_base.cc:1034] running on GCE node
W20260812 14:47:22.841691 20857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:22.842180 20848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:22.842429 20848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:22.843590 20848 hybrid_clock.cc:648] HybridClock initialized: now 1786546042843578 us; error 30 us; skew 500 ppm
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:22.846402 20848 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:22.846997 20848 webserver.cc:460] Webserver started at http://127.13.146.130:36829/ using document root <none> and password file <none>
I20260812 14:47:22.847267 20848 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:22.847345 20848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:22.847520 20848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:22.848680 20848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "167323f4145d4b1e90115b57fd95defe"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "ceddc9ec1642dce43f5b238023557ff1"
server_key_iv: "6b71122b30ac1cc59ae9c61b54049198"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.849097 20848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "167323f4145d4b1e90115b57fd95defe"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "ceddc9ec1642dce43f5b238023557ff1"
server_key_iv: "6b71122b30ac1cc59ae9c61b54049198"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.850613 20848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:22.851423 20875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.851603 20848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:22.851715 20848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "167323f4145d4b1e90115b57fd95defe"
format_stamp: "Formatted at 2026-08-12 14:47:22 on dist-test-slave-8hhm"
server_key: "ceddc9ec1642dce43f5b238023557ff1"
server_key_iv: "6b71122b30ac1cc59ae9c61b54049198"
server_key_version: "encryptionkey@0"
I20260812 14:47:22.851816 20848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:22.870302 20848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:22.872566 20848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:22.872741 20848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:22.873018 20848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:22.873436 20848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:22.873495 20848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.873543 20848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:22.873587 20848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:22.880127 20848 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:43153
I20260812 14:47:22.880203 21099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:43153 every 8 connection(s)
I20260812 14:47:22.880578 20848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:22.884137 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20848
I20260812 14:47:22.884275 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:22.884506 13898 external_mini_cluster.cc:1428] Setting key e4f7e3c63c68f6ce157109aa097f55db
Aug 12 14:47:22 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546042, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:47:22.896631 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:22.881767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:49359 (local address 127.13.146.190:34973)
0812 14:47:22.881853 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:22.881856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:22.884958 (+ 3102us) server_negotiation.cc:407] Connection header received
0812 14:47:22.886485 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:22.886486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:22.886501 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:22.886587 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:22.887637 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.888459 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.890208 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.890622 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.890721 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:22.893518 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:22.893520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:22.893520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:22.893529 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:22.895093 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.895543 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.895544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.895544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.895608 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:22.895904 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:22.895904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:22.895905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:22.896028 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:22.896103 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:22.896426 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 14:47:22.896520 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:47:22.896860 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:22.881754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34973 (local address 127.13.146.130:49359)
0812 14:47:22.884796 (+ 3042us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:22.884851 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:22.886355 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:22.886722 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:22.886727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:22.887167 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:22.887498 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:22.887505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.888609 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.888610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:22.890056 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:22.890059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:22.890740 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:22.890741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:22.890917 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:22.891502 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:22.891523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:22.893416 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:22.895200 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:22.895209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:22.895212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:22.895456 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:22.895678 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:22.895679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:22.895679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:22.895825 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:22.896108 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:22.896115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:22.896277 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 14:47:22.896389 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:47:22.896601 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2990,"thread_start_us":93,"threads_started":1}
I20260812 14:47:22.897393 21100 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34973
I20260812 14:47:22.897526 21100 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:22.897775 21100 heartbeater.cc:511] Master 127.13.146.190:34973 requested a full tablet report, sending...
I20260812 14:47:22.898240 20478 ts_manager.cc:194] Registered new tserver with Master: 167323f4145d4b1e90115b57fd95defe (127.13.146.130:43153)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
I20260812 14:47:22.898727 20478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:49359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:22.916493 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:22.999903 21111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:23.000177 21111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:23.000216 21111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:23.001392 21111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:23.001434 21111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:23.001461 21111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:23.003098 21111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:23.003356 21111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:23.003589 21111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:23.008955 21128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:23.008950 21125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:23.008956 21124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:23.009279 21111 server_base.cc:1034] running on GCE node
I20260812 14:47:23.009482 21111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:23.009702 21111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:23.010859 21111 hybrid_clock.cc:648] HybridClock initialized: now 1786546043010853 us; error 40 us; skew 500 ppm
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546043, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:23.013533 21111 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:23.014002 21111 webserver.cc:460] Webserver started at http://127.13.146.131:33119/ using document root <none> and password file <none>
I20260812 14:47:23.014259 21111 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:23.014330 21111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:23.014479 21111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:23.015573 21111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "64be1bd779214f9b9e351418a3977779"
format_stamp: "Formatted at 2026-08-12 14:47:23 on dist-test-slave-8hhm"
server_key: "268d97a637a7d2de5dd91946e1067fc9"
server_key_iv: "512135c4d9f61d9dfc057c196e0dceec"
server_key_version: "encryptionkey@0"
I20260812 14:47:23.015947 21111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "64be1bd779214f9b9e351418a3977779"
format_stamp: "Formatted at 2026-08-12 14:47:23 on dist-test-slave-8hhm"
server_key: "268d97a637a7d2de5dd91946e1067fc9"
server_key_iv: "512135c4d9f61d9dfc057c196e0dceec"
server_key_version: "encryptionkey@0"
I20260812 14:47:23.017388 21111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:47:23.018157 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:23.018347 21111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:23.018464 21111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "64be1bd779214f9b9e351418a3977779"
format_stamp: "Formatted at 2026-08-12 14:47:23 on dist-test-slave-8hhm"
server_key: "268d97a637a7d2de5dd91946e1067fc9"
server_key_iv: "512135c4d9f61d9dfc057c196e0dceec"
server_key_version: "encryptionkey@0"
I20260812 14:47:23.018545 21111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:23.031380 21111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:23.033854 21111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:23.034024 21111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:23.034271 21111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:23.034584 21111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:23.034637 21111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:23.034679 21111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:23.034708 21111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:23.040645 21111 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:39955
I20260812 14:47:23.040686 21351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:39955 every 8 connection(s)
I20260812 14:47:23.041039 21111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:23.042042 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21111
I20260812 14:47:23.042150 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:23.042356 13898 external_mini_cluster.cc:1428] Setting key 0ca7bd8c1d8df8f477f3336ccb2c55e3
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546043, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:23.050699 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:23.041968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:45415 (local address 127.13.146.190:34973)
0812 14:47:23.042024 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:23.042027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:23.042230 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:47:23.043559 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:23.043560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:23.043571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:23.043634 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:23.044223 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.044717 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.045815 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.045998 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.046053 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:23.048230 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:23.048232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:23.048232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:23.048239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:23.049541 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.049902 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.049902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.049903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.049956 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.050182 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.050183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.050183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.050283 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:23.050334 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:23.050555 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 14:47:23.050633 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:47:23.050879 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:23.041850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34973 (local address 127.13.146.131:45415)
0812 14:47:23.042078 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:23.042134 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:23.043466 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:23.043695 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:23.043698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:23.043950 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:23.044147 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:23.044153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.044816 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.044817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:23.045741 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:23.045743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.046093 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.046094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:23.046195 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:23.046673 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:23.046681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:23.048149 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:23.049620 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:23.049626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:23.049628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:23.049839 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:23.050011 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:23.050011 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:23.050011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:23.050130 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:23.050336 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:23.050340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:23.050448 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 14:47:23.050529 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:47:23.050648 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260812 14:47:23.051460 21352 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34973
I20260812 14:47:23.051589 21352 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:23.051810 21352 heartbeater.cc:511] Master 127.13.146.190:34973 requested a full tablet report, sending...
I20260812 14:47:23.052270 20478 ts_manager.cc:194] Registered new tserver with Master: 64be1bd779214f9b9e351418a3977779 (127.13.146.131:39955)
I20260812 14:47:23.052672 20478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:45415
I20260812 14:47:23.054144 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:23.078881 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:23.070774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.13.146.129:39231)
0812 14:47:23.070938 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:23.070950 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:23.070962 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:23.070992 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:23.070994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:23.071009 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:23.071080 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:23.071647 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.072375 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.073098 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.073536 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.073604 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:23.075952 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:23.075962 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:23.075962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:23.075969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:23.077604 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.077986 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.077995 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.077995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.078070 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.078279 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.078280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.078280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.078428 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:23.078614 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:23.078669 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 14:47:23.078749 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[20400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:23.087209 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:23.080415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39336 (local address 127.13.146.190:34973)
0812 14:47:23.080470 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:23.080472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:23.080483 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:23.080562 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:23.080563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:23.080573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:23.080656 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:23.081153 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.081721 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.082404 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:23.082619 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:23.082696 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:23.084891 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:23.084893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:23.084893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:23.084899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:23.086171 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.086513 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.086514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.086514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.086566 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:23.086743 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:23.086743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:23.086744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:23.086887 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:23.086964 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:23.087075 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:47:23.087143 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 14:47:23.087494 20478 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39336
I20260812 14:47:23.087877 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20587
I20260812 14:47:23.101606 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20848
I20260812 14:47:23.124200 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21111
I20260812 14:47:23.141058 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20421
2026-08-12T14:47:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2208 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[21379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[21379](info): set up 2 sockets
Aug 12 14:47:23 dist-test-slave-8hhm krb5kdc[21379](info): commencing operation
krb5kdc: starting...
W20260812 14:47:24.460721 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:24 dist-test-slave-8hhm krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:24.588181 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45321
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:24.674368 21404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:24.674669 21404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:24.674714 21404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:24.675918 21404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:24.676041 21404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:24.676071 21404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:24.676097 21404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:24.676131 21404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:24.677733 21404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:24.678054 21404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:24.678306 21404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:24.683758 21420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:24.683754 21416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:24.683754 21417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:24.683771 21404 server_base.cc:1034] running on GCE node
I20260812 14:47:24.684405 21404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:24.684747 21404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:24.685915 21404 hybrid_clock.cc:648] HybridClock initialized: now 1786546044685890 us; error 36 us; skew 500 ppm
Aug 12 14:47:24 dist-test-slave-8hhm krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546044, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:24.688663 21404 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:24.689042 21404 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:24.689095 21404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:24.689190 21404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:24.690342 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "2de07ed2955652a8fb79f81148508e7b"
server_key_iv: "9cf85860632554d18398647968e7feb3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.690672 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "2de07ed2955652a8fb79f81148508e7b"
server_key_iv: "9cf85860632554d18398647968e7feb3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.691985 21404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:24.692742 21435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:24.692975 21404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:24.693078 21404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "2de07ed2955652a8fb79f81148508e7b"
server_key_iv: "9cf85860632554d18398647968e7feb3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.693145 21404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:24.700659 21404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:24.702625 21404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:24.702737 21404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:24.706856 21404 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:45321
I20260812 14:47:24.706934 21525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:45321 every 8 connection(s)
I20260812 14:47:24.707216 21404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:24.708343 21526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:24.710906 21526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: Bootstrap starting.
I20260812 14:47:24.711552 21526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:24.711891 21526 log.cc:824] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:24.712596 21526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: No bootstrap required, opened a new log
I20260812 14:47:24.713747 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21404
I20260812 14:47:24.713860 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:24.714072 13898 external_mini_cluster.cc:1428] Setting key 07ca54f8bf7c7882d153d23b627aa451
I20260812 14:47:24.714745 21526 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } }
I20260812 14:47:24.714964 21526 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:24.715000 21526 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e14c9cc61d41a1bfaf2ee28662bc42, State: Initialized, Role: FOLLOWER
I20260812 14:47:24.715180 21526 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } }
I20260812 14:47:24.715281 21526 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:24.715322 21526 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:24.715369 21526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:24.716018 21526 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } }
I20260812 14:47:24.716123 21526 leader_election.cc:302] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e14c9cc61d41a1bfaf2ee28662bc42; no voters:
I20260812 14:47:24.716317 21526 leader_election.cc:288] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:24.716390 21536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:24.716535 21536 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [term 1 LEADER]: Becoming Leader. State: Replica: c1e14c9cc61d41a1bfaf2ee28662bc42, State: Running, Role: LEADER
I20260812 14:47:24.716574 21526 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:24.716667 21536 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } }
I20260812 14:47:24.717003 21538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } } }
I20260812 14:47:24.716992 21536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e14c9cc61d41a1bfaf2ee28662bc42. Latest consensus state: current_term: 1 leader_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e14c9cc61d41a1bfaf2ee28662bc42" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45321 } } }
I20260812 14:47:24.717196 21536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:24.717121 21538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:24.717499 21552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:24.718112 21552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:24.720069 21552 catalog_manager.cc:1347] Generated new cluster ID: 9fe365ee297c43e9866f5381918862c9
I20260812 14:47:24.720120 21552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:24 dist-test-slave-8hhm krb5kdc[21379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:24.725415 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:24.714650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.13.146.190:45321)
0812 14:47:24.714858 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:24.714868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:24.714903 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:47:24.715346 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:24.715356 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:24.715423 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:24.715594 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:24.716209 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.717777 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.718926 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.719379 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.719490 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:24.722030 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:24.722035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:24.722035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:24.722055 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:24.723785 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:24.724179 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:24.724182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:24.724182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:24.724289 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:24.724515 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:24.724516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:24.724516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:24.724765 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:24.724971 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:24.725040 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 14:47:24.725189 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260812 14:47:24.741150 21552 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:24.741843 21552 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:24.749298 21552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: Generated new TSK 0
I20260812 14:47:24.749526 21552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:24.784742 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:24.877336 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:24.877619 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:24.877665 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:24.878872 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:24.878953 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:24.879002 21578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:24.880594 21578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:24.880879 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:24.881163 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:24.888130 21591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:24.888160 21598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:24.888161 21590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:24.890137 21578 server_base.cc:1034] running on GCE node
I20260812 14:47:24.890445 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:24.890741 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:24.891922 21578 hybrid_clock.cc:648] HybridClock initialized: now 1786546044891892 us; error 45 us; skew 500 ppm
Aug 12 14:47:24 dist-test-slave-8hhm krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546044, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:24.894687 21578 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:24.895102 21578 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:24.895182 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:24.895342 21578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:24.896512 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "3e2008229f804ccdad61f4e45a925e33"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "24b86a98e8ae49ac402d00c240d57074"
server_key_iv: "6f00f05ee8699c51ff855fa15f2d99c3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.896905 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "3e2008229f804ccdad61f4e45a925e33"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "24b86a98e8ae49ac402d00c240d57074"
server_key_iv: "6f00f05ee8699c51ff855fa15f2d99c3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.898334 21578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:24.899101 21606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:24.899279 21578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:24.899385 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "3e2008229f804ccdad61f4e45a925e33"
format_stamp: "Formatted at 2026-08-12 14:47:24 on dist-test-slave-8hhm"
server_key: "24b86a98e8ae49ac402d00c240d57074"
server_key_iv: "6f00f05ee8699c51ff855fa15f2d99c3"
server_key_version: "encryptionkey@0"
I20260812 14:47:24.899470 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:24.910416 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:24.912928 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:24.913108 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:24.913431 21578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:24.913769 21578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:24.913825 21578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:24.913890 21578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:24.913920 21578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:24.920418 21578 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44183
I20260812 14:47:24.920465 21803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44183 every 8 connection(s)
I20260812 14:47:24.920749 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:24.921608 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21578
I20260812 14:47:24.921737 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:24.921962 13898 external_mini_cluster.cc:1428] Setting key 0e9240b2c28463866a072ae86aff5a5e
Aug 12 14:47:24 dist-test-slave-8hhm krb5kdc[21379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546044, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
I20260812 14:47:24.932488 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:24.921623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:45039 (local address 127.13.146.190:45321)
0812 14:47:24.921710 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:24.921713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:24.921977 (+ 264us) server_negotiation.cc:407] Connection header received
0812 14:47:24.922627 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:24.922628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:24.922646 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:24.922732 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:24.924071 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.924666 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.926229 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.926438 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.926514 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:24.929309 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:24.929310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:24.929311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:24.929322 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:24.930871 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:24.931391 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:24.931392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:24.931393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:24.931450 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:24.931766 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:24.931767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:24.931767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:24.931876 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:24.931929 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:24.932319 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 14:47:24.932420 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:47:24.932756 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:24.921593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45321 (local address 127.13.146.129:45039)
0812 14:47:24.921814 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:24.921869 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:24.922501 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:24.922826 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:24.922831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:24.923234 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:24.923938 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:24.923946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.924758 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.924760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:24.926123 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:24.926127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:24.926537 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:24.926538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:24.926689 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:24.927256 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:24.927266 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:24.929159 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:24.930966 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:24.930974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:24.930976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:24.931305 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:24.931527 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:24.931528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:24.931528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:24.931683 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:24.931949 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:24.931956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:24.932133 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 14:47:24.932261 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 14:47:24.932447 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":134,"threads_started":1}
I20260812 14:47:24.933475 21808 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45321
I20260812 14:47:24.933586 21808 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:24.933722 21808 heartbeater.cc:511] Master 127.13.146.190:45321 requested a full tablet report, sending...
I20260812 14:47:24.934315 21462 ts_manager.cc:194] Registered new tserver with Master: 3e2008229f804ccdad61f4e45a925e33 (127.13.146.129:44183)
I20260812 14:47:24.934973 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:45039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:24.952586 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:25.038241 21816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:25.038496 21816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:25.038537 21816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:25.039714 21816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:25.039788 21816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:25.039832 21816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:25.041414 21816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:25.041693 21816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:25.041935 21816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:25.047645 21822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.047618 21828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.047607 21834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.047863 21816 server_base.cc:1034] running on GCE node
I20260812 14:47:25.048178 21816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:25.048493 21816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:25.049671 21816 hybrid_clock.cc:648] HybridClock initialized: now 1786546045049645 us; error 40 us; skew 500 ppm
Aug 12 14:47:25 dist-test-slave-8hhm krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546045, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:25.052305 21816 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:25.052682 21816 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:25.052758 21816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:25.052912 21816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:25.054067 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "5dc5186562ad44f1b795525ba7d47e8f"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "0cb222e5f25e8a1c3b674dc578bade03"
server_key_iv: "7850d361a1263bd6592964ff1ba244c0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.054450 21816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "5dc5186562ad44f1b795525ba7d47e8f"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "0cb222e5f25e8a1c3b674dc578bade03"
server_key_iv: "7850d361a1263bd6592964ff1ba244c0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.055892 21816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:25.056794 21845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.057024 21816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:25.057169 21816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "5dc5186562ad44f1b795525ba7d47e8f"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "0cb222e5f25e8a1c3b674dc578bade03"
server_key_iv: "7850d361a1263bd6592964ff1ba244c0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.057303 21816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:25.074257 21816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:25.076594 21816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:25.076820 21816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:25.077121 21816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:25.077561 21816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:25.077630 21816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.077682 21816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:25.077739 21816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.085073 21816 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34279
I20260812 14:47:25.085141 22050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34279 every 8 connection(s)
I20260812 14:47:25.085502 21816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:25.088528 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21816
I20260812 14:47:25.088624 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:25.088835 13898 external_mini_cluster.cc:1428] Setting key 269808cfd874a036114d67ef5290f429
Aug 12 14:47:25 dist-test-slave-8hhm krb5kdc[21379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546045, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:25.095791 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.086321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:33781 (local address 127.13.146.190:45321)
0812 14:47:25.086379 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.086381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.086586 (+ 205us) server_negotiation.cc:407] Connection header received
0812 14:47:25.087102 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.087104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.087116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.087195 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.088192 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.088767 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.090061 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.090255 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.090306 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.092667 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.092669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.092669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:25.092676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.094193 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.094648 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.094649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.094649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.094712 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.095031 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.095032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.095032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.095154 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.095209 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.095639 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.095722 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:25.096071 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.086280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45321 (local address 127.13.146.130:33781)
0812 14:47:25.086461 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.086507 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.086995 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.087276 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.087280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.087554 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:25.088072 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.088078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.088862 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.088864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.089970 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.089973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.090336 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.090337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.090485 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.091004 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:25.091015 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:25.092570 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:25.094285 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:25.094293 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:25.094296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:25.094536 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:25.094789 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:25.094790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:25.094790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:25.094956 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:25.095225 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:25.095233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:25.095441 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.095566 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.095772 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260812 14:47:25.096691 22053 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45321
I20260812 14:47:25.096794 22053 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:25.096978 22053 heartbeater.cc:511] Master 127.13.146.190:45321 requested a full tablet report, sending...
I20260812 14:47:25.097488 21462 ts_manager.cc:194] Registered new tserver with Master: 5dc5186562ad44f1b795525ba7d47e8f (127.13.146.130:34279)
I20260812 14:47:25.097831 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:33781
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:25.115901 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:25.200273 22062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:25.200527 22062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:25.200569 22062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:25.201895 22062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:25.201962 22062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:25.202013 22062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:25.203689 22062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:45321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:25.203975 22062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:25.204216 22062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:25.209662 22076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.209592 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.209609 22062 server_base.cc:1034] running on GCE node
W20260812 14:47:25.209592 22080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.210116 22062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:25.210410 22062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:25.211578 22062 hybrid_clock.cc:648] HybridClock initialized: now 1786546045211541 us; error 52 us; skew 500 ppm
Aug 12 14:47:25 dist-test-slave-8hhm krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546045, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:25.214262 22062 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:25.214574 22062 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:25.214620 22062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:25.214710 22062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:25.215763 22062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "b258d7d0022844f89333f67b8937b16a"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "f303ae6a6a84f01ba63bfaf0ccb118ac"
server_key_iv: "8c1adb5b13cff13e469e7af68e36c4a0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.216084 22062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "b258d7d0022844f89333f67b8937b16a"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "f303ae6a6a84f01ba63bfaf0ccb118ac"
server_key_iv: "8c1adb5b13cff13e469e7af68e36c4a0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.217617 22062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:25.218360 22092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.218602 22062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:25.218683 22062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "b258d7d0022844f89333f67b8937b16a"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "f303ae6a6a84f01ba63bfaf0ccb118ac"
server_key_iv: "8c1adb5b13cff13e469e7af68e36c4a0"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.218736 22062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:25.245831 22062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:25.248298 22062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:25.248440 22062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:25.248656 22062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:25.249022 22062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:25.249069 22062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.249094 22062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:25.249142 22062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.255371 22062 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:39359
I20260812 14:47:25.255447 22288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:39359 every 8 connection(s)
I20260812 14:47:25.255746 22062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:25.261838 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22062
I20260812 14:47:25.261950 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:25.262151 13898 external_mini_cluster.cc:1428] Setting key d929844040aeda318c11d0dae69b3286
Aug 12 14:47:25 dist-test-slave-8hhm krb5kdc[21379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546045, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:25.265547 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.256555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:52209 (local address 127.13.146.190:45321)
0812 14:47:25.256625 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.256627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.256840 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:47:25.257400 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.257402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.257412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.257477 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.258451 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.258938 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.259924 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.260176 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.260232 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.262895 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.262896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.262897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:25.262906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.264264 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.264652 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.264653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.264653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.264707 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.264936 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.264936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.264936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.265048 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.265098 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.265417 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.265482 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:47:25.265769 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.256513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45321 (local address 127.13.146.131:52209)
0812 14:47:25.256708 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.256752 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.257300 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.257560 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.257564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.257848 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:25.258368 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.258374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.259018 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.259019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.259843 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.259845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.260260 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.260260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.260364 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.260900 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:25.260909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:25.262781 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:25.264337 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:25.264343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:25.264345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:25.264574 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:25.264768 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:25.264768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:25.264769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:25.264876 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:25.265104 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:25.265109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:25.265292 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.265397 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.265526 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260812 14:47:25.266331 22290 heartbeater.cc:347] Connected to a master server at 127.13.146.190:45321
I20260812 14:47:25.266422 22290 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:25.266541 22290 heartbeater.cc:511] Master 127.13.146.190:45321 requested a full tablet report, sending...
I20260812 14:47:25.266866 21462 ts_manager.cc:194] Registered new tserver with Master: b258d7d0022844f89333f67b8937b16a (127.13.146.131:39359)
I20260812 14:47:25.267262 21462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:52209
I20260812 14:47:25.273936 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:47:25.281054 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.275329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.13.146.190:45321)
0812 14:47:25.275379 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.275381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.275460 (+ 79us) server_negotiation.cc:407] Connection header received
0812 14:47:25.275510 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.275511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.275520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.275567 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.276158 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.276638 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.277405 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.277595 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.277653 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.278837 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.278839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.278839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:25.278849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.280069 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.280395 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.280396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.280396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.280454 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:25.280631 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:25.280631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:25.280632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:25.280768 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.280842 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.280927 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.280988 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:25.282939 21462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:25.284690 21462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:25.291410 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34279 (local address 127.0.0.1:56124)
0812 14:47:25.287420 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.287462 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.287530 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.287856 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.287860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.287862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:25.287942 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.287944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.289298 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.289299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.290399 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.290401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.291089 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.291184 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291197 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.291273 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.291329 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":5,"thread_start_us":88,"threads_started":1}
I20260812 14:47:25.291419 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.13.146.130:34279)
0812 14:47:25.287517 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.287521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.287531 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:25.287648 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.287650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.287667 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.287750 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:25.288026 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.289167 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.290497 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.290995 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291091 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291160 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.291297 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.291335 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":87,"threads_started":1}
I20260812 14:47:25.291509 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44183 (local address 127.0.0.1:35756)
0812 14:47:25.287690 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.287698 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.287767 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.288133 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.288133 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.288135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:25.288170 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.288171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.289031 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.289032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.290354 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.290357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.291288 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.291339 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291343 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.291388 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.291433 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":71,"threads_started":1}
I20260812 14:47:25.291780 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.13.146.129:44183)
0812 14:47:25.287635 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.287639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.287838 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:47:25.287953 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.287956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.287973 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.288044 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:25.288246 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.288928 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.290436 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.291146 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291211 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291259 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.291634 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.291689 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260812 14:47:25.291780 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.13.146.131:39359)
0812 14:47:25.287439 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.287445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.287528 (+ 83us) server_negotiation.cc:407] Connection header received
0812 14:47:25.287652 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.287654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.287669 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.287750 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:25.288031 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.289164 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.290346 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.291075 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291154 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291213 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.291422 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.291466 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":98,"threads_started":1}
I20260812 14:47:25.292265 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.287062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:39359 (local address 127.0.0.1:36814)
0812 14:47:25.287416 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.287460 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.287560 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.287864 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.287865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.287866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:25.287943 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.287944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.289314 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.289315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.290253 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.290256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.291252 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.291253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.291320 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.291325 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.292140 (+ 815us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.292188 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":5,"thread_start_us":78,"threads_started":1}
I20260812 14:47:25.292272 21932 tablet_service.cc:1467] Processing CreateTablet for tablet 6e52bf7f904b410584ba72705c089b9e (DEFAULT_TABLE table=test-table [id=46422e8ef97449cdb0762fc7463b5092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:25.292690 21932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e52bf7f904b410584ba72705c089b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:25.292958 22169 tablet_service.cc:1467] Processing CreateTablet for tablet 6e52bf7f904b410584ba72705c089b9e (DEFAULT_TABLE table=test-table [id=46422e8ef97449cdb0762fc7463b5092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:25.293318 22169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e52bf7f904b410584ba72705c089b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:25.295606 22319 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Bootstrap starting.
I20260812 14:47:25.296135 21687 tablet_service.cc:1467] Processing CreateTablet for tablet 6e52bf7f904b410584ba72705c089b9e (DEFAULT_TABLE table=test-table [id=46422e8ef97449cdb0762fc7463b5092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:25.296312 22319 tablet_bootstrap.cc:654] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:25.296244 22320 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Bootstrap starting.
I20260812 14:47:25.296507 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e52bf7f904b410584ba72705c089b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:25.296936 22320 tablet_bootstrap.cc:654] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:25.296928 22319 log.cc:824] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:25.297670 22320 log.cc:824] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:25.297922 22319 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: No bootstrap required, opened a new log
I20260812 14:47:25.298002 22319 ts_tablet_manager.cc:1397] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:25.298388 22320 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: No bootstrap required, opened a new log
I20260812 14:47:25.298542 22320 ts_tablet_manager.cc:1397] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:25.300010 22325 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Bootstrap starting.
I20260812 14:47:25.300124 22320 raft_consensus.cc:348] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.300298 22320 raft_consensus.cc:374] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:25.300349 22320 raft_consensus.cc:729] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b258d7d0022844f89333f67b8937b16a, State: Initialized, Role: FOLLOWER
I20260812 14:47:25.300494 22320 consensus_queue.cc:260] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.300501 22319 raft_consensus.cc:348] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.300699 22319 raft_consensus.cc:374] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:25.300746 22319 raft_consensus.cc:729] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc5186562ad44f1b795525ba7d47e8f, State: Initialized, Role: FOLLOWER
I20260812 14:47:25.300791 22320 ts_tablet_manager.cc:1428] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:25.300913 22325 tablet_bootstrap.cc:654] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:25.300891 22319 consensus_queue.cc:260] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.300940 22290 heartbeater.cc:503] Master 127.13.146.190:45321 was elected leader, sending a full tablet report...
I20260812 14:47:25.301182 22319 ts_tablet_manager.cc:1428] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:25.301249 22053 heartbeater.cc:503] Master 127.13.146.190:45321 was elected leader, sending a full tablet report...
I20260812 14:47:25.301556 22325 log.cc:824] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:25.302363 22325 tablet_bootstrap.cc:492] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: No bootstrap required, opened a new log
I20260812 14:47:25.302448 22325 ts_tablet_manager.cc:1397] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 14:47:25.304006 22325 raft_consensus.cc:348] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.304134 22325 raft_consensus.cc:374] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:25.304157 22325 raft_consensus.cc:729] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2008229f804ccdad61f4e45a925e33, State: Initialized, Role: FOLLOWER
I20260812 14:47:25.304298 22325 consensus_queue.cc:260] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.304559 22325 ts_tablet_manager.cc:1428] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:47:25.304594 21808 heartbeater.cc:503] Master 127.13.146.190:45321 was elected leader, sending a full tablet report...
W20260812 14:47:25.336095 22055 tablet.cc:2369] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:47:25.421641 21809 tablet.cc:2369] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:25.456017 22327 raft_consensus.cc:482] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:25.456182 22327 raft_consensus.cc:504] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.456725 22327 leader_election.cc:288] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e2008229f804ccdad61f4e45a925e33 (127.13.146.129:44183), 5dc5186562ad44f1b795525ba7d47e8f (127.13.146.130:34279)
I20260812 14:47:25.459585 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.457020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:43097 (local address 127.13.146.129:44183)
0812 14:47:25.457106 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.457117 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.457130 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:25.457162 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.457163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.457173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.457268 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:25.457424 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.457889 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.458857 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.459219 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.459305 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.459313 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.459480 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.459519 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 14:47:25.459585 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.456954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44183 (local address 127.13.146.131:43097)
0812 14:47:25.457019 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.457042 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.457090 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.457275 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.457276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.457278 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:25.457352 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.457354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.457976 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.457977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.458746 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.458749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.459302 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.459303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.459404 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.459406 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.459474 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.459513 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 14:47:25.459820 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.456977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34279 (local address 127.13.146.131:40419)
0812 14:47:25.457201 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.457252 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.457336 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.457490 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.457490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.457492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:25.457577 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.457579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.458224 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.458225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.458973 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:25.458975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.459541 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.459542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:25.459638 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.459642 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.459696 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.459744 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":124,"threads_started":1}
I20260812 14:47:25.459870 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.457128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:40419 (local address 127.13.146.130:34279)
0812 14:47:25.457178 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.457181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.457320 (+ 139us) server_negotiation.cc:407] Connection header received
0812 14:47:25.457393 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.457394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.457403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.457473 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:25.457647 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.458130 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.459081 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.459455 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.459540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.459548 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.459752 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.459802 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:25.459985 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e52bf7f904b410584ba72705c089b9e" candidate_uuid: "b258d7d0022844f89333f67b8937b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2008229f804ccdad61f4e45a925e33" is_pre_election: true
I20260812 14:47:25.460175 21720 raft_consensus.cc:2454] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b258d7d0022844f89333f67b8937b16a in term 0.
I20260812 14:47:25.460167 21969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e52bf7f904b410584ba72705c089b9e" candidate_uuid: "b258d7d0022844f89333f67b8937b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc5186562ad44f1b795525ba7d47e8f" is_pre_election: true
I20260812 14:47:25.460331 21969 raft_consensus.cc:2454] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b258d7d0022844f89333f67b8937b16a in term 0.
I20260812 14:47:25.460500 22095 leader_election.cc:302] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e2008229f804ccdad61f4e45a925e33, b258d7d0022844f89333f67b8937b16a; no voters:
I20260812 14:47:25.460667 22327 raft_consensus.cc:2781] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:25.460726 22327 raft_consensus.cc:482] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:25.460788 22327 raft_consensus.cc:3037] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:25.461663 22327 raft_consensus.cc:504] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.461814 22327 leader_election.cc:288] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [CANDIDATE]: Term 1 election: Requested vote from peers 3e2008229f804ccdad61f4e45a925e33 (127.13.146.129:44183), 5dc5186562ad44f1b795525ba7d47e8f (127.13.146.130:34279)
I20260812 14:47:25.461988 21969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e52bf7f904b410584ba72705c089b9e" candidate_uuid: "b258d7d0022844f89333f67b8937b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc5186562ad44f1b795525ba7d47e8f"
I20260812 14:47:25.461995 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e52bf7f904b410584ba72705c089b9e" candidate_uuid: "b258d7d0022844f89333f67b8937b16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2008229f804ccdad61f4e45a925e33"
I20260812 14:47:25.462080 21969 raft_consensus.cc:3037] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:25.462075 21720 raft_consensus.cc:3037] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:25.462805 21969 raft_consensus.cc:2454] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b258d7d0022844f89333f67b8937b16a in term 1.
I20260812 14:47:25.462818 21720 raft_consensus.cc:2454] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b258d7d0022844f89333f67b8937b16a in term 1.
I20260812 14:47:25.463030 22096 leader_election.cc:302] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc5186562ad44f1b795525ba7d47e8f, b258d7d0022844f89333f67b8937b16a; no voters:
I20260812 14:47:25.463151 22327 raft_consensus.cc:2781] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:25.463318 22327 raft_consensus.cc:686] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 1 LEADER]: Becoming Leader. State: Replica: b258d7d0022844f89333f67b8937b16a, State: Running, Role: LEADER
I20260812 14:47:25.463433 22327 consensus_queue.cc:237] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } }
I20260812 14:47:25.464305 21460 catalog_manager.cc:5495] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a reported cstate change: term changed from 0 to 1, leader changed from <none> to b258d7d0022844f89333f67b8937b16a (127.13.146.131). New cstate: current_term: 1 leader_uuid: "b258d7d0022844f89333f67b8937b16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b258d7d0022844f89333f67b8937b16a" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 39359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:25.472054 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.469855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.13.146.131:39359)
0812 14:47:25.469908 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.469910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.469917 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:47:25.469946 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.469946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.469961 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.470017 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:25.470148 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.470617 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.471380 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:25.471557 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:25.471610 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:25.471779 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:25.471872 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:25.471933 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.471966 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.471982 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:47:25.473963 21969 raft_consensus.cc:1261] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 1 FOLLOWER]: Refusing update from remote peer b258d7d0022844f89333f67b8937b16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:25.473963 21720 raft_consensus.cc:1261] T 6e52bf7f904b410584ba72705c089b9e P 3e2008229f804ccdad61f4e45a925e33 [term 1 FOLLOWER]: Refusing update from remote peer b258d7d0022844f89333f67b8937b16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:25.474370 22334 consensus_queue.cc:1035] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e2008229f804ccdad61f4e45a925e33" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:25.474493 22327 consensus_queue.cc:1035] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc5186562ad44f1b795525ba7d47e8f" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:25.477387 22343 mvcc.cc:205] Tried to move back new op lower bound from 7317692602258546688 to 7317692602218848256. Current Snapshot: MvccSnapshot[applied={T|T < 7317692602258546688}]
I20260812 14:47:25.479586 21462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:25.479683 21462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:25.480434 21462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1e14c9cc61d41a1bfaf2ee28662bc42: Sending DeleteTablet for 3 replicas of tablet 6e52bf7f904b410584ba72705c089b9e
I20260812 14:47:25.480822 21932 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e52bf7f904b410584ba72705c089b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:25 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56124
I20260812 14:47:25.480860 22169 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e52bf7f904b410584ba72705c089b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:25 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260812 14:47:25.481089 22356 tablet_replica.cc:330] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: stopping tablet replica
I20260812 14:47:25.481158 21687 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e52bf7f904b410584ba72705c089b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:25 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:35756
I20260812 14:47:25.481199 22356 raft_consensus.cc:2229] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:25.481314 22357 tablet_replica.cc:330] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: stopping tablet replica
I20260812 14:47:25.481364 22356 raft_consensus.cc:2258] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:25.481513 22357 raft_consensus.cc:2229] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:25.481575 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21578
I20260812 14:47:25.495611 22356 ts_tablet_manager.cc:1905] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:25.498329 22356 ts_tablet_manager.cc:1918] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:25.498454 22356 log.cc:1196] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/6e52bf7f904b410584ba72705c089b9e
I20260812 14:47:25.498828 22356 ts_tablet_manager.cc:1939] T 6e52bf7f904b410584ba72705c089b9e P b258d7d0022844f89333f67b8937b16a: Deleting consensus metadata
I20260812 14:47:25.499470 21438 catalog_manager.cc:4841] TS b258d7d0022844f89333f67b8937b16a (127.13.146.131:39359): tablet 6e52bf7f904b410584ba72705c089b9e (table test-table [id=46422e8ef97449cdb0762fc7463b5092]) successfully deleted
I20260812 14:47:25.501629 22357 raft_consensus.cc:2258] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:25.502295 22357 ts_tablet_manager.cc:1905] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:25.504041 22357 ts_tablet_manager.cc:1918] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:25.504230 22357 log.cc:1196] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/6e52bf7f904b410584ba72705c089b9e
I20260812 14:47:25.504585 22357 ts_tablet_manager.cc:1939] T 6e52bf7f904b410584ba72705c089b9e P 5dc5186562ad44f1b795525ba7d47e8f: Deleting consensus metadata
I20260812 14:47:25.505155 21439 catalog_manager.cc:4841] TS 5dc5186562ad44f1b795525ba7d47e8f (127.13.146.130:34279): tablet 6e52bf7f904b410584ba72705c089b9e (table test-table [id=46422e8ef97449cdb0762fc7463b5092]) successfully deleted
W20260812 14:47:25.505913 21438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:44183 (error 108)
I20260812 14:47:25.506155 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21816
I20260812 14:47:25.538834 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.538593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44183 (local address 127.0.0.1:35768)
0812 14:47:25.538693 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.538780 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 14:47:25.539072 21438 catalog_manager.cc:4584] TS 3e2008229f804ccdad61f4e45a925e33 (127.13.146.129:44183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e52bf7f904b410584ba72705c089b9e: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44183: connect: Connection refused (error 111)
I20260812 14:47:25.542692 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22062
I20260812 14:47:25.558975 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21404
2026-08-12T14:47:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2420 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T14:47:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:25Z Disabled control of system clock
I20260812 14:47:25.596133 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39773
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:39773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:47:25.683584 22373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:25.683794 22373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:25.683815 22373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:25.684998 22373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:25.685045 22373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:25.685065 22373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:25.685077 22373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:25.685089 22373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:25.686543 22373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:39773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:25.686813 22373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:25.687129 22373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:47:25.692379 22373 server_base.cc:1034] running on GCE node
W20260812 14:47:25.692346 22384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.692324 22382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.692346 22388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.692833 22373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:25.693182 22373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:25.694403 22373 hybrid_clock.cc:648] HybridClock initialized: now 1786546045694375 us; error 45 us; skew 500 ppm
I20260812 14:47:25.695811 22373 webserver.cc:460] Webserver started at http://127.13.146.190:33017/ using document root <none> and password file <none>
I20260812 14:47:25.696074 22373 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:25.696159 22373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:25.696311 22373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:25.697482 22373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "5b486fe64fbb4e0f8203c14fd80f5dba"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "52474e9e7042ff3bb7ca738956638deb"
server_key_iv: "222af8c76ecc1c5bfee00c70a8a87b5d"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.697868 22373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "5b486fe64fbb4e0f8203c14fd80f5dba"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "52474e9e7042ff3bb7ca738956638deb"
server_key_iv: "222af8c76ecc1c5bfee00c70a8a87b5d"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.699288 22373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:25.700191 22397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.700428 22373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:25.700542 22373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "5b486fe64fbb4e0f8203c14fd80f5dba"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "52474e9e7042ff3bb7ca738956638deb"
server_key_iv: "222af8c76ecc1c5bfee00c70a8a87b5d"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.700634 22373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:25.704192 22373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:25.704485 22373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:25.704641 22373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:25.708972 22373 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:39773
I20260812 14:47:25.709005 22489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:39773 every 8 connection(s)
I20260812 14:47:25.709419 22373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:25.710139 22491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:25.710865 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22373
I20260812 14:47:25.710995 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:25.711262 13898 external_mini_cluster.cc:1428] Setting key 786d64b45a68d5119de059a37c49a7c1
I20260812 14:47:25.714145 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.711856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.13.146.190:39773)
0812 14:47:25.712054 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.712063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.712088 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:47:25.712551 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.712572 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.712615 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.712736 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.713615 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.713619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.713619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:25.713623 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.713677 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:25.713688 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.713753 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.713852 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.713936 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260812 14:47:25.714613 22491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: Bootstrap starting.
I20260812 14:47:25.715255 22491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:25.715492 22491 log.cc:824] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:25.716154 22491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: No bootstrap required, opened a new log
I20260812 14:47:25.717631 22491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } }
I20260812 14:47:25.717775 22491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:25.717799 22491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b486fe64fbb4e0f8203c14fd80f5dba, State: Initialized, Role: FOLLOWER
I20260812 14:47:25.717917 22491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } }
I20260812 14:47:25.718015 22491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:25.718039 22491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:25.718066 22491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:25.718536 22491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } }
I20260812 14:47:25.718622 22491 leader_election.cc:302] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b486fe64fbb4e0f8203c14fd80f5dba; no voters:
I20260812 14:47:25.718750 22491 leader_election.cc:288] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:25.719002 22491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:25.719120 22497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:25.719300 22497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [term 1 LEADER]: Becoming Leader. State: Replica: 5b486fe64fbb4e0f8203c14fd80f5dba, State: Running, Role: LEADER
I20260812 14:47:25.719412 22497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } }
I20260812 14:47:25.719859 22498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } } }
I20260812 14:47:25.719871 22500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b486fe64fbb4e0f8203c14fd80f5dba. Latest consensus state: current_term: 1 leader_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b486fe64fbb4e0f8203c14fd80f5dba" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39773 } } }
I20260812 14:47:25.719959 22498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:25.719970 22500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:25.720158 22515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:25.720674 22515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:25.722339 22515 catalog_manager.cc:1347] Generated new cluster ID: f9f8883a046f494881f785ba4eb6e168
I20260812 14:47:25.722394 22515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:25.730880 22515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:25.731132 22515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:25.737138 22515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: Generated new TSK 0
I20260812 14:47:25.737398 22515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:25.739362 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--builtin_ntp_servers=127.13.146.148:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:47:25.828647 22533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:25.828881 22533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:25.828917 22533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:25.830139 22533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:25.830211 22533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:25.830256 22533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:25.831621 22533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:25.831912 22533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:25.832228 22533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:25.838286 22545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.838286 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.838500 22533 server_base.cc:1034] running on GCE node
W20260812 14:47:25.838320 22541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.838828 22533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:25.839102 22533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:25.840296 22533 hybrid_clock.cc:648] HybridClock initialized: now 1786546045840280 us; error 30 us; skew 500 ppm
I20260812 14:47:25.841737 22533 webserver.cc:460] Webserver started at http://127.13.146.129:37431/ using document root <none> and password file <none>
I20260812 14:47:25.842036 22533 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:25.842118 22533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:25.842273 22533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:25.843408 22533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "c822d5318df4422f8b57cb3bf02d301b"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "2f95ccc18fc7ae7bd0256f099a2f539f"
server_key_iv: "b996464086c66de8db2e6026b0ed7bbe"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.843811 22533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "c822d5318df4422f8b57cb3bf02d301b"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "2f95ccc18fc7ae7bd0256f099a2f539f"
server_key_iv: "b996464086c66de8db2e6026b0ed7bbe"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.845314 22533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:25.846184 22556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.846386 22533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:25.846524 22533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "c822d5318df4422f8b57cb3bf02d301b"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "2f95ccc18fc7ae7bd0256f099a2f539f"
server_key_iv: "b996464086c66de8db2e6026b0ed7bbe"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.846621 22533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:25.850234 22533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:25.850517 22533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:25.850663 22533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:25.850884 22533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:25.851202 22533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:25.851257 22533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.851296 22533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:25.851323 22533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.857185 22533 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:46311
I20260812 14:47:25.857277 22769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:46311 every 8 connection(s)
I20260812 14:47:25.857614 22533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:25.858582 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22533
I20260812 14:47:25.858691 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:25.858914 13898 external_mini_cluster.cc:1428] Setting key 05bfe6eba5ed8451fa0f4523b00579b5
I20260812 14:47:25.859947 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--builtin_ntp_servers=127.13.146.148:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 14:47:25.861317 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.858393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:55765 (local address 127.13.146.190:39773)
0812 14:47:25.858475 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.858478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.858770 (+ 292us) server_negotiation.cc:407] Connection header received
0812 14:47:25.859477 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.859478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.859485 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.859535 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.861009 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.861010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.861010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:25.861014 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.861028 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:25.861033 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.861092 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.861196 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.861208 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:47:25.861620 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.858357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39773 (local address 127.13.146.129:55765)
0812 14:47:25.858617 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.858669 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.859370 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.859608 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.859612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.859622 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:25.860839 (+ 1217us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:25.860840 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:25.860879 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:25.860913 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:25.861115 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:25.861120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:25.861123 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.861198 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.861284 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":120,"threads_started":1}
I20260812 14:47:25.862336 22770 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39773
I20260812 14:47:25.862442 22770 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:25.862596 22770 heartbeater.cc:511] Master 127.13.146.190:39773 requested a full tablet report, sending...
I20260812 14:47:25.863163 22423 ts_manager.cc:194] Registered new tserver with Master: c822d5318df4422f8b57cb3bf02d301b (127.13.146.129:46311)
W20260812 14:47:25.949883 22780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:25.950122 22780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:25.950158 22780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:25.951575 22780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:25.951619 22780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:25.951646 22780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:25.952975 22780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:25.953182 22780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:25.953511 22780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:25.958510 22794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.958520 22797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:25.958544 22791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:25.958890 22780 server_base.cc:1034] running on GCE node
I20260812 14:47:25.959146 22780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:25.959393 22780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:25.960541 22780 hybrid_clock.cc:648] HybridClock initialized: now 1786546045960508 us; error 47 us; skew 500 ppm
I20260812 14:47:25.961874 22780 webserver.cc:460] Webserver started at http://127.13.146.130:40961/ using document root <none> and password file <none>
I20260812 14:47:25.962157 22780 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:25.962206 22780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:25.962373 22780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:25.963442 22780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "e881bf847ee643fb98a9f44cac061fc6"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "a1441a13d21c8dc466fd49a22fb6efda"
server_key_iv: "0908c586c2f4f67a247076dba81092ca"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.963816 22780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "e881bf847ee643fb98a9f44cac061fc6"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "a1441a13d21c8dc466fd49a22fb6efda"
server_key_iv: "0908c586c2f4f67a247076dba81092ca"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.965409 22780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:25.966255 22807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.966455 22780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:25.966593 22780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "e881bf847ee643fb98a9f44cac061fc6"
format_stamp: "Formatted at 2026-08-12 14:47:25 on dist-test-slave-8hhm"
server_key: "a1441a13d21c8dc466fd49a22fb6efda"
server_key_iv: "0908c586c2f4f67a247076dba81092ca"
server_key_version: "encryptionkey@0"
I20260812 14:47:25.966672 22780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:25.970081 22780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:25.970333 22780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:25.970474 22780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:25.970701 22780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:25.970952 22780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:25.970981 22780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.971001 22780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:25.971015 22780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:25.976603 22780 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:38337
I20260812 14:47:25.976748 23019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:38337 every 8 connection(s)
I20260812 14:47:25.976996 22780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:25.980046 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.977736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:33945 (local address 127.13.146.190:39773)
0812 14:47:25.977807 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:25.977809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:25.977993 (+ 184us) server_negotiation.cc:407] Connection header received
0812 14:47:25.978504 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:25.978505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:25.978511 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:25.978580 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:25.979817 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:25.979818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:25.979819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:25.979823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:25.979838 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:25.979842 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:25.979916 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:25.979964 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:47:25.979976 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:47:25.980247 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:25.977726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39773 (local address 127.13.146.130:33945)
0812 14:47:25.977895 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:25.977926 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:25.978391 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:25.978634 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:25.978637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:25.978645 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:25.979657 (+ 1012us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:25.979658 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:25.979686 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:25.979713 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:25.979916 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:25.979918 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:25.979920 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:25.979970 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:47:25.980017 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260812 14:47:25.980710 23022 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39773
I20260812 14:47:25.981035 23022 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:25.981271 23022 heartbeater.cc:511] Master 127.13.146.190:39773 requested a full tablet report, sending...
I20260812 14:47:25.981685 22423 ts_manager.cc:194] Registered new tserver with Master: e881bf847ee643fb98a9f44cac061fc6 (127.13.146.130:38337)
I20260812 14:47:25.986986 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22780
I20260812 14:47:25.987085 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:25.987295 13898 external_mini_cluster.cc:1428] Setting key 8b6e3039f836a7ee4cd76388059cc5f0
I20260812 14:47:25.988284 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--builtin_ntp_servers=127.13.146.148:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:47:26.072088 23027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:26.072306 23027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:26.072330 23027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:26.073451 23027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:26.073488 23027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:26.073513 23027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:26.074812 23027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:26.075011 23027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:26.075253 23027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:26.080411 23044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.080389 23039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.080389 23038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:26.080744 23027 server_base.cc:1034] running on GCE node
I20260812 14:47:26.080936 23027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:26.081256 23027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:26.082439 23027 hybrid_clock.cc:648] HybridClock initialized: now 1786546046082426 us; error 33 us; skew 500 ppm
I20260812 14:47:26.083846 23027 webserver.cc:460] Webserver started at http://127.13.146.131:37671/ using document root <none> and password file <none>
I20260812 14:47:26.084177 23027 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:26.084234 23027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:26.084388 23027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:26.085675 23027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "aedbf7c7c5d14c42898fbed2ec959d1b"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "91f114d4b9d2ff20f2ba6aade5bdea86"
server_key_iv: "7f0e8b35811af796c1b5598732a6d80e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.086064 23027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "aedbf7c7c5d14c42898fbed2ec959d1b"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "91f114d4b9d2ff20f2ba6aade5bdea86"
server_key_iv: "7f0e8b35811af796c1b5598732a6d80e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.087587 23027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:26.088439 23057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.088692 23027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:26.088810 23027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "aedbf7c7c5d14c42898fbed2ec959d1b"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "91f114d4b9d2ff20f2ba6aade5bdea86"
server_key_iv: "7f0e8b35811af796c1b5598732a6d80e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.088884 23027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:26.092146 23027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:26.092361 23027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:26.092456 23027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:26.092612 23027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:26.092913 23027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:26.092969 23027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.093012 23027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:26.093040 23027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.099045 23027 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:34071
I20260812 14:47:26.099083 23278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:34071 every 8 connection(s)
I20260812 14:47:26.099419 23027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:26.102289 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.100155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:46681 (local address 127.13.146.190:39773)
0812 14:47:26.100220 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.100222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.100440 (+ 218us) server_negotiation.cc:407] Connection header received
0812 14:47:26.100901 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.100902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.100907 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.100961 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.102084 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.102086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.102086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.102090 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.102103 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.102108 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.102165 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.102226 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.102238 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:47:26.102466 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.100138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39773 (local address 127.13.146.131:46681)
0812 14:47:26.100343 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.100375 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.100817 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.101013 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.101016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.101024 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.101949 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.101950 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.101977 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.102003 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.102166 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.102169 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.102171 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.102224 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.102270 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260812 14:47:26.102984 23280 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39773
I20260812 14:47:26.103096 23280 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:26.103219 23280 heartbeater.cc:511] Master 127.13.146.190:39773 requested a full tablet report, sending...
I20260812 14:47:26.103530 22423 ts_manager.cc:194] Registered new tserver with Master: aedbf7c7c5d14c42898fbed2ec959d1b (127.13.146.131:34071)
I20260812 14:47:26.103715 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23027
I20260812 14:47:26.103822 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:26.104017 13898 external_mini_cluster.cc:1428] Setting key bbdb3efe93f8d50ad8904087cf97c0ac
I20260812 14:47:26.104801 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:47:26.107275 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.106155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.13.146.190:39773)
0812 14:47:26.106189 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.106191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.106320 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:47:26.106359 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.106360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.106364 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.106414 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.107123 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.107124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.107125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.107128 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.107139 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.107143 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.107188 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.107222 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.107232 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:47:26.108955 22423 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:26.110620 22423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:26.114573 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.112950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.13.146.129:46311)
0812 14:47:26.113325 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.113331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.113368 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:47:26.113432 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.113435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.113446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.113762 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.114185 (+ 423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.114190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.114191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.114195 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.114234 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.114274 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.114346 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.114446 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.114465 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":84,"threads_started":1}
I20260812 14:47:26.114720 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.113035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.13.146.130:38337)
0812 14:47:26.113300 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.113304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.113315 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:26.113342 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.113344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.113353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.113452 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.114304 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.114313 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.114313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.114319 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.114361 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.114415 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.114520 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.114591 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.114611 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":129,"threads_started":1}
I20260812 14:47:26.114751 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.112771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:38337 (local address 127.0.0.1:56434)
0812 14:47:26.112973 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.113017 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.113074 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.113457 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.113463 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.113471 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.114186 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.114186 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.114194 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.114209 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.114518 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.114519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.114520 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.114566 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.114586 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260812 14:47:26.114751 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.112776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46311 (local address 127.0.0.1:52232)
0812 14:47:26.113296 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.113308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.113368 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.113580 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.113581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.113585 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.114059 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.114059 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.114077 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.114102 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.114338 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.114346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.114349 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.114419 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.114434 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":53,"threads_started":1}
I20260812 14:47:26.115258 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.112781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:34071 (local address 127.0.0.1:35482)
0812 14:47:26.113855 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.113868 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.113928 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.114353 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.114354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.114356 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.114871 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.114871 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.114878 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.114892 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.115128 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.115128 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.115129 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.115183 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.115196 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":112,"threads_started":1}
I20260812 14:47:26.115283 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.113070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.13.146.131:34071)
0812 14:47:26.113293 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.113298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.113947 (+ 649us) server_negotiation.cc:407] Connection header received
0812 14:47:26.113979 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.113982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.113992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.114080 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.114978 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.114985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.114985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.114989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.115021 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.115047 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.115118 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.115192 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.115206 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":106,"threads_started":1}
I20260812 14:47:26.115608 22898 tablet_service.cc:1467] Processing CreateTablet for tablet d8f89010618243a7b475686e840116c2 (DEFAULT_TABLE table=test-table [id=6c720c6bd7c34a5e880676541c89a847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:26.115720 22653 tablet_service.cc:1467] Processing CreateTablet for tablet d8f89010618243a7b475686e840116c2 (DEFAULT_TABLE table=test-table [id=6c720c6bd7c34a5e880676541c89a847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:26.115833 23152 tablet_service.cc:1467] Processing CreateTablet for tablet d8f89010618243a7b475686e840116c2 (DEFAULT_TABLE table=test-table [id=6c720c6bd7c34a5e880676541c89a847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:26.116005 22898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f89010618243a7b475686e840116c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.116032 22653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f89010618243a7b475686e840116c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.116098 23152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8f89010618243a7b475686e840116c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.118839 23307 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Bootstrap starting.
I20260812 14:47:26.119537 23307 tablet_bootstrap.cc:654] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.119719 23314 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Bootstrap starting.
I20260812 14:47:26.120101 23307 log.cc:824] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.120102 23315 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Bootstrap starting.
I20260812 14:47:26.120507 23314 tablet_bootstrap.cc:654] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.120821 23315 tablet_bootstrap.cc:654] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.120872 23314 log.cc:824] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.121160 23307 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: No bootstrap required, opened a new log
I20260812 14:47:26.121243 23315 log.cc:824] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.121271 23307 ts_tablet_manager.cc:1397] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:26.121735 23314 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: No bootstrap required, opened a new log
I20260812 14:47:26.121821 23314 ts_tablet_manager.cc:1397] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:26.122148 23315 tablet_bootstrap.cc:492] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: No bootstrap required, opened a new log
I20260812 14:47:26.122252 23315 ts_tablet_manager.cc:1397] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:47:26.124044 23314 raft_consensus.cc:348] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.124193 23314 raft_consensus.cc:374] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.124248 23314 raft_consensus.cc:729] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c822d5318df4422f8b57cb3bf02d301b, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.124145 23307 raft_consensus.cc:348] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.124321 23307 raft_consensus.cc:374] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.124377 23307 raft_consensus.cc:729] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aedbf7c7c5d14c42898fbed2ec959d1b, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.124403 23314 consensus_queue.cc:260] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.124522 23307 consensus_queue.cc:260] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.124734 23314 ts_tablet_manager.cc:1428] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:26.124810 23307 ts_tablet_manager.cc:1428] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:26.124876 22770 heartbeater.cc:503] Master 127.13.146.190:39773 was elected leader, sending a full tablet report...
I20260812 14:47:26.124922 23280 heartbeater.cc:503] Master 127.13.146.190:39773 was elected leader, sending a full tablet report...
I20260812 14:47:26.125061 23315 raft_consensus.cc:348] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.125258 23315 raft_consensus.cc:374] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.125303 23315 raft_consensus.cc:729] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e881bf847ee643fb98a9f44cac061fc6, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.125422 23315 consensus_queue.cc:260] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.125680 23315 ts_tablet_manager.cc:1428] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:26.125747 23022 heartbeater.cc:503] Master 127.13.146.190:39773 was elected leader, sending a full tablet report...
I20260812 14:47:26.148826 23325 raft_consensus.cc:482] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:26.148938 23325 raft_consensus.cc:504] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.149294 23325 leader_election.cc:288] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c822d5318df4422f8b57cb3bf02d301b (127.13.146.129:46311), e881bf847ee643fb98a9f44cac061fc6 (127.13.146.130:38337)
I20260812 14:47:26.150771 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.149473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:38337 (local address 127.13.146.131:49483)
0812 14:47:26.149526 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.149539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.149576 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.149725 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.149725 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.149728 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.150284 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.150284 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.150291 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.150303 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.150609 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.150610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.150611 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.150660 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.150674 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:47:26.150825 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.149537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:49483 (local address 127.13.146.130:38337)
0812 14:47:26.149591 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.149593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.149602 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:26.149629 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.149630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.149636 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.149697 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.150378 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.150380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.150381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.150386 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.150406 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.150411 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.150480 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.150734 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.150750 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:47:26.150995 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.149546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:43719 (local address 127.13.146.129:46311)
0812 14:47:26.149613 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.149616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.149781 (+ 165us) server_negotiation.cc:407] Connection header received
0812 14:47:26.149811 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.149812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.149818 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.149886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.150713 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.150714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.150714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.150719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.150734 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.150738 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.150809 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.150897 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.150914 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:26.151216 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.149480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46311 (local address 127.13.146.131:43719)
0812 14:47:26.149697 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.149706 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.149754 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.149895 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.149895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.149897 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:26.150578 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:26.150579 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:26.150585 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:26.150596 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:26.150808 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:26.150809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:26.150809 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 14:47:26.151123 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 14:47:26.151136 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":59,"thread_start_us":106,"threads_started":1}
I20260812 14:47:26.151189 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f89010618243a7b475686e840116c2" candidate_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e881bf847ee643fb98a9f44cac061fc6" is_pre_election: true
I20260812 14:47:26.151341 22938 raft_consensus.cc:2454] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aedbf7c7c5d14c42898fbed2ec959d1b in term 0.
I20260812 14:47:26.151579 23064 leader_election.cc:302] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aedbf7c7c5d14c42898fbed2ec959d1b, e881bf847ee643fb98a9f44cac061fc6; no voters:
I20260812 14:47:26.151654 22688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f89010618243a7b475686e840116c2" candidate_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c822d5318df4422f8b57cb3bf02d301b" is_pre_election: true
I20260812 14:47:26.151748 23325 raft_consensus.cc:2781] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:26.151854 22688 raft_consensus.cc:2454] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aedbf7c7c5d14c42898fbed2ec959d1b in term 0.
I20260812 14:47:26.151886 23325 raft_consensus.cc:482] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:26.151926 23325 raft_consensus.cc:3037] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.152738 23325 raft_consensus.cc:504] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.152884 23325 leader_election.cc:288] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [CANDIDATE]: Term 1 election: Requested vote from peers c822d5318df4422f8b57cb3bf02d301b (127.13.146.129:46311), e881bf847ee643fb98a9f44cac061fc6 (127.13.146.130:38337)
I20260812 14:47:26.153040 22938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f89010618243a7b475686e840116c2" candidate_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e881bf847ee643fb98a9f44cac061fc6"
I20260812 14:47:26.153108 22688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f89010618243a7b475686e840116c2" candidate_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c822d5318df4422f8b57cb3bf02d301b"
I20260812 14:47:26.153146 22938 raft_consensus.cc:3037] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.153185 22688 raft_consensus.cc:3037] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.154129 22688 raft_consensus.cc:2454] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aedbf7c7c5d14c42898fbed2ec959d1b in term 1.
I20260812 14:47:26.154129 22938 raft_consensus.cc:2454] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aedbf7c7c5d14c42898fbed2ec959d1b in term 1.
I20260812 14:47:26.154291 23064 leader_election.cc:302] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aedbf7c7c5d14c42898fbed2ec959d1b, e881bf847ee643fb98a9f44cac061fc6; no voters:
I20260812 14:47:26.154474 23325 raft_consensus.cc:2781] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:26.154652 23325 raft_consensus.cc:686] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 1 LEADER]: Becoming Leader. State: Replica: aedbf7c7c5d14c42898fbed2ec959d1b, State: Running, Role: LEADER
I20260812 14:47:26.154731 23325 consensus_queue.cc:237] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } }
I20260812 14:47:26.155527 22421 catalog_manager.cc:5495] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to aedbf7c7c5d14c42898fbed2ec959d1b (127.13.146.131). New cstate: current_term: 1 leader_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aedbf7c7c5d14c42898fbed2ec959d1b" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 34071 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:26.164628 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.163772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.13.146.131:34071)
0812 14:47:26.163834 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.163840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.163849 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:26.163916 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.163916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.163928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.163982 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.164460 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.164461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.164461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.164465 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.164478 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.164481 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.164532 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.164567 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.164578 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:47:26.166541 22938 raft_consensus.cc:1261] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 1 FOLLOWER]: Refusing update from remote peer aedbf7c7c5d14c42898fbed2ec959d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:26.166543 22688 raft_consensus.cc:1261] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 1 FOLLOWER]: Refusing update from remote peer aedbf7c7c5d14c42898fbed2ec959d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:26.166946 23330 consensus_queue.cc:1035] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c822d5318df4422f8b57cb3bf02d301b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:26.167107 23325 consensus_queue.cc:1035] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e881bf847ee643fb98a9f44cac061fc6" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:26.171550 22423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:26.171656 22423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:26.172420 22423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b486fe64fbb4e0f8203c14fd80f5dba: Sending DeleteTablet for 3 replicas of tablet d8f89010618243a7b475686e840116c2
I20260812 14:47:26.172748 22653 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f89010618243a7b475686e840116c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:26 UTC) from {username='slave'} at 127.0.0.1:52232
I20260812 14:47:26.172778 22898 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f89010618243a7b475686e840116c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:26 UTC) from {username='slave'} at 127.0.0.1:56434
I20260812 14:47:26.173005 23355 tablet_replica.cc:330] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: stopping tablet replica
I20260812 14:47:26.173121 23355 raft_consensus.cc:2229] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:26.173256 23355 raft_consensus.cc:2258] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:26.173262 23152 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f89010618243a7b475686e840116c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:26 UTC) from {username='slave'} at 127.0.0.1:35482
I20260812 14:47:26.173416 23349 tablet_replica.cc:330] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: stopping tablet replica
I20260812 14:47:26.173473 23356 tablet_replica.cc:330] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: stopping tablet replica
I20260812 14:47:26.173514 23349 raft_consensus.cc:2229] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:26.173578 23356 raft_consensus.cc:2229] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:26.173614 23349 raft_consensus.cc:2258] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:26.173671 23356 raft_consensus.cc:2258] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:26.173203 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22533
I20260812 14:47:26.174062 23349 ts_tablet_manager.cc:1905] T d8f89010618243a7b475686e840116c2 P c822d5318df4422f8b57cb3bf02d301b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:26.174103 23356 ts_tablet_manager.cc:1905] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:26.175539 23356 ts_tablet_manager.cc:1918] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:26.175631 23356 log.cc:1196] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/d8f89010618243a7b475686e840116c2
I20260812 14:47:26.175915 23356 ts_tablet_manager.cc:1939] T d8f89010618243a7b475686e840116c2 P aedbf7c7c5d14c42898fbed2ec959d1b: Deleting consensus metadata
I20260812 14:47:26.176329 22398 catalog_manager.cc:4841] TS aedbf7c7c5d14c42898fbed2ec959d1b (127.13.146.131:34071): tablet d8f89010618243a7b475686e840116c2 (table test-table [id=6c720c6bd7c34a5e880676541c89a847]) successfully deleted
I20260812 14:47:26.178285 23355 ts_tablet_manager.cc:1905] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:26.179889 23355 ts_tablet_manager.cc:1918] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:26.179973 23355 log.cc:1196] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/d8f89010618243a7b475686e840116c2
I20260812 14:47:26.180279 23355 ts_tablet_manager.cc:1939] T d8f89010618243a7b475686e840116c2 P e881bf847ee643fb98a9f44cac061fc6: Deleting consensus metadata
I20260812 14:47:26.180696 22405 catalog_manager.cc:4841] TS e881bf847ee643fb98a9f44cac061fc6 (127.13.146.130:38337): tablet d8f89010618243a7b475686e840116c2 (table test-table [id=6c720c6bd7c34a5e880676541c89a847]) successfully deleted
W20260812 14:47:26.185262 22400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:46311 (error 108)
I20260812 14:47:26.185513 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22780
I20260812 14:47:26.188314 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.185665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46311 (local address 127.0.0.1:52236)
0812 14:47:26.188219 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.188273 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2516}
W20260812 14:47:26.195156 22400 catalog_manager.cc:4584] TS c822d5318df4422f8b57cb3bf02d301b (127.13.146.129:46311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8f89010618243a7b475686e840116c2: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46311: connect: Connection refused (error 111)
I20260812 14:47:26.210192 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23027
I20260812 14:47:26.227496 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22373
2026-08-12T14:47:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (669 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 14:47:26.257292 13898 mini_oidc.cc:152] Starting JWKS server
I20260812 14:47:26.258494 13898 webserver.cc:460] Webserver started at https://127.0.0.1:33823/ using document root <none> and password file <none>
I20260812 14:47:26.258576 13898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:47:26.258713 13898 webserver.cc:460] Webserver started at http://127.0.0.1:43597/ using document root <none> and password file <none>
2026-08-12T14:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:26Z Disabled control of system clock
I20260812 14:47:26.268808 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:32921
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:32921
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33823/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/testchainca.cert with env {}
W20260812 14:47:26.351452 23374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:26.351672 23374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:26.351696 23374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:26.352918 23374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:26.352962 23374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:26.352974 23374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:26.352986 23374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:26.353025 23374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:47:26.353052 23374 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:47:26.353068 23374 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33823/jwks.json
I20260812 14:47:26.354664 23374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:32921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:32921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33823/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:26.354921 23374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:26.355147 23374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:26.360076 23394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.360049 23386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.360268 23389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:26.360177 23374 server_base.cc:1034] running on GCE node
I20260812 14:47:26.360559 23374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:26.360858 23374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:26.362035 23374 hybrid_clock.cc:648] HybridClock initialized: now 1786546046362014 us; error 36 us; skew 500 ppm
I20260812 14:47:26.363251 23374 webserver.cc:460] Webserver started at http://127.13.146.190:45335/ using document root <none> and password file <none>
I20260812 14:47:26.363480 23374 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:26.363525 23374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:26.363613 23374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:26.364607 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "bcf014888cb848818478d8ded8596890"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "4a903c3ac92a08021646b5ba50b5fbd4"
server_key_iv: "00917006ba77c1971c018b6d57499019"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.364944 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "bcf014888cb848818478d8ded8596890"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "4a903c3ac92a08021646b5ba50b5fbd4"
server_key_iv: "00917006ba77c1971c018b6d57499019"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.366274 23374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:26.367002 23408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.367172 23374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:26.367262 23374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "bcf014888cb848818478d8ded8596890"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "4a903c3ac92a08021646b5ba50b5fbd4"
server_key_iv: "00917006ba77c1971c018b6d57499019"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.367318 23374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:26.392604 23374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:26.392910 23374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:26.393020 23374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:26.396920 23374 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:32921
I20260812 14:47:26.397020 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:32921 every 8 connection(s)
I20260812 14:47:26.397368 23374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:26.398420 23513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.400817 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890: Bootstrap starting.
I20260812 14:47:26.401506 23513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.401846 23513 log.cc:824] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.402693 23513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890: No bootstrap required, opened a new log
I20260812 14:47:26.404090 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23374
I20260812 14:47:26.404211 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:26.404415 13898 external_mini_cluster.cc:1428] Setting key 60ba1610e30022283c6c9f907a9fd1fe
I20260812 14:47:26.404727 23513 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } }
I20260812 14:47:26.404897 23513 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.404938 23513 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf014888cb848818478d8ded8596890, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.405067 23513 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } }
I20260812 14:47:26.405156 23513 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:26.405203 23513 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:26.405274 23513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.405810 23513 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } }
I20260812 14:47:26.405926 23513 leader_election.cc:302] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcf014888cb848818478d8ded8596890; no voters:
I20260812 14:47:26.406139 23513 leader_election.cc:288] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:26.406168 23526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:26.406287 23526 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [term 1 LEADER]: Becoming Leader. State: Replica: bcf014888cb848818478d8ded8596890, State: Running, Role: LEADER
I20260812 14:47:26.406385 23526 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } }
I20260812 14:47:26.406418 23513 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:26.406725 23529 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcf014888cb848818478d8ded8596890. Latest consensus state: current_term: 1 leader_uuid: "bcf014888cb848818478d8ded8596890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } } }
I20260812 14:47:26.406816 23529 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.406965 23527 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcf014888cb848818478d8ded8596890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf014888cb848818478d8ded8596890" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 32921 } } }
I20260812 14:47:26.407019 23527 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.407122 23536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:26.407721 23536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:26.409798 23536 catalog_manager.cc:1347] Generated new cluster ID: 43a136dbb5bb4ae0a8fdd71a1efe5bed
I20260812 14:47:26.409857 23536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:26.410979 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.404957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.13.146.190:32921)
0812 14:47:26.405243 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.405252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.405287 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:47:26.405415 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.405420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.405473 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.405613 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.405859 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.407673 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.408781 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.409425 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.409576 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.410342 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.410346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.410346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.410528 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.410561 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.410571 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.410644 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.410728 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.410807 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260812 14:47:26.411878 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:47:26.428722 23536 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:26.429437 23536 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:26.435427 23536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcf014888cb848818478d8ded8596890: Generated new TSK 0
I20260812 14:47:26.435627 23536 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:47:26.469165 23572 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 14:47:26.469420 23572 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 14:47:26.469444 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.446549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.13.146.190:32921)
0812 14:47:26.446600 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.446602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.446721 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:47:26.446848 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.446849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.446863 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.446948 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:47:26.447119 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.447629 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.448407 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.448618 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.448694 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.448925 (+ 231us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:47:26.469119 (+ 20194us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:47:26.469368 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 14:47:26.469522 23522 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 14:47:26.470052 23570 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:26.446470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:32921 (local address 127.0.0.1:45088)
0812 14:47:26.446650 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.446659 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.446745 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.446958 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.446959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.446961 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:47:26.447039 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.447042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.447717 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.447719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.448319 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.448321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.448708 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.448710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.448773 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.448810 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:47:26.469291 (+ 20481us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:47:26.469720 (+ 429us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.146.190:32921: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.146.190:32921: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 14:47:26.470665 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23374
2026-08-12T14:47:26Z chronyd exiting
I20260812 14:47:26.524919 13898 test_util.cc:175] -----------------------------------------------
I20260812 14:47:26.525004 13898 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786546020513802-13898-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (268 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 14:47:26.525847 13898 mini_oidc.cc:152] Starting JWKS server
I20260812 14:47:26.526599 13898 webserver.cc:460] Webserver started at https://127.0.0.1:35947/ using document root <none> and password file <none>
I20260812 14:47:26.526643 13898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:47:26.526808 13898 webserver.cc:460] Webserver started at http://127.0.0.1:34449/ using document root <none> and password file <none>
2026-08-12T14:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:26Z Disabled control of system clock
I20260812 14:47:26.535722 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35559
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:44495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:35559
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35947/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/testchainca.cert with env {}
W20260812 14:47:26.617499 23583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:26.617699 23583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:26.617722 23583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:26.618877 23583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:26.618921 23583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:26.618934 23583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:26.618947 23583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:26.619004 23583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:47:26.619025 23583 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:47:26.619050 23583 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35947/jwks.json
I20260812 14:47:26.620573 23583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:35559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35947/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:26.620834 23583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:26.621052 23583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:26.626194 23596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.626223 23594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.626403 23593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:26.626505 23583 server_base.cc:1034] running on GCE node
I20260812 14:47:26.626760 23583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:26.627049 23583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:26.628221 23583 hybrid_clock.cc:648] HybridClock initialized: now 1786546046628199 us; error 51 us; skew 500 ppm
I20260812 14:47:26.629504 23583 webserver.cc:460] Webserver started at http://127.13.146.190:41305/ using document root <none> and password file <none>
I20260812 14:47:26.629766 23583 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:26.629839 23583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:26.629988 23583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:26.631023 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "8515f27487c5445e8586969ca3ac6387"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "925e4866e42d505fddd85034fc6bf5ea"
server_key_iv: "59d17e26dce405ece6c3e87bc296b50c"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.631402 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "8515f27487c5445e8586969ca3ac6387"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "925e4866e42d505fddd85034fc6bf5ea"
server_key_iv: "59d17e26dce405ece6c3e87bc296b50c"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.632803 23583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:26.633626 23604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.633797 23583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.633894 23583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "8515f27487c5445e8586969ca3ac6387"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "925e4866e42d505fddd85034fc6bf5ea"
server_key_iv: "59d17e26dce405ece6c3e87bc296b50c"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.633981 23583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:26.641040 23583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:26.641402 23583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:26.641562 23583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:26.645714 23583 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:35559
I20260812 14:47:26.645761 23710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:35559 every 8 connection(s)
I20260812 14:47:26.646092 23583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:26.646796 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.649703 23711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: Bootstrap starting.
I20260812 14:47:26.650004 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23583
I20260812 14:47:26.650141 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:26.650327 13898 external_mini_cluster.cc:1428] Setting key b874624cce077a75f7f27a1ed641dfc0
I20260812 14:47:26.650445 23711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.650825 23711 log.cc:824] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.651649 23711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: No bootstrap required, opened a new log
I20260812 14:47:26.653983 23711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } }
I20260812 14:47:26.654177 23711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.654215 23711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8515f27487c5445e8586969ca3ac6387, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.654369 23711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } }
I20260812 14:47:26.654443 23711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:26.654466 23711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:26.654554 23711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.655278 23711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } }
I20260812 14:47:26.655413 23711 leader_election.cc:302] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8515f27487c5445e8586969ca3ac6387; no voters:
I20260812 14:47:26.655594 23711 leader_election.cc:288] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:26.655623 23717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:26.655836 23717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [term 1 LEADER]: Becoming Leader. State: Replica: 8515f27487c5445e8586969ca3ac6387, State: Running, Role: LEADER
I20260812 14:47:26.655872 23711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:26.655942 23717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } }
I20260812 14:47:26.656281 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8515f27487c5445e8586969ca3ac6387. Latest consensus state: current_term: 1 leader_uuid: "8515f27487c5445e8586969ca3ac6387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } } }
I20260812 14:47:26.656361 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.656626 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.650886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.13.146.190:35559)
0812 14:47:26.651118 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.651127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.651161 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:47:26.651692 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.651700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.651802 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.651943 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.652202 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.653070 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.654307 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.654858 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.654952 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.655788 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.655792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.655793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.656077 (+ 284us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.656129 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.656144 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.656226 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.656334 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.656421 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260812 14:47:26.656607 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8515f27487c5445e8586969ca3ac6387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8515f27487c5445e8586969ca3ac6387" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35559 } } }
I20260812 14:47:26.656931 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.657732 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 14:47:26.657845 23753 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:26.657892 23753 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:26.657967 23755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:26.658200 23755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:26.659817 23755 catalog_manager.cc:1347] Generated new cluster ID: e810bc4fc07040e3b2ef873ece4fe897
I20260812 14:47:26.659878 23755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:26.667892 23755 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:26.668589 23755 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:26.672500 23755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8515f27487c5445e8586969ca3ac6387: Generated new TSK 0
I20260812 14:47:26.672690 23755 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:47:26.693544 23769 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 14:47:26.693634 23769 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 14:47:26.693964 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.678720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.13.146.190:35559)
0812 14:47:26.678794 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.678796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.678856 (+ 60us) server_negotiation.cc:407] Connection header received
0812 14:47:26.678921 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.678922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.678934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.679022 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:47:26.679170 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.679957 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.680728 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.681047 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.681120 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.681310 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:47:26.693746 (+ 12436us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:47:26.693891 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 14:47:26.694027 23715 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 14:47:26.694054 23767 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:26.678608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35559 (local address 127.0.0.1:41248)
0812 14:47:26.678798 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.678808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.678862 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.679027 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.679027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.679029 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:47:26.679081 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.679082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.680054 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.680054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.680634 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.680636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.681128 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.681129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.681168 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.681173 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:47:26.693865 (+ 12692us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:47:26.693950 (+ 85us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.146.190:35559: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260812 14:47:26.694425 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23583
2026-08-12T14:47:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 14:47:26.746577 13898 mini_oidc.cc:152] Starting JWKS server
I20260812 14:47:26.747335 13898 webserver.cc:460] Webserver started at https://127.0.0.1:41201/ using document root <none> and password file <none>
I20260812 14:47:26.747411 13898 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:47:26.747601 13898 webserver.cc:460] Webserver started at http://127.0.0.1:45151/ using document root <none> and password file <none>
2026-08-12T14:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:26Z Disabled control of system clock
I20260812 14:47:26.756673 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35359
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:35359
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:47:26.840529 23782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:26.840794 23782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:26.840832 23782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:26.842067 23782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:26.842140 23782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:26.842161 23782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:26.842182 23782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:26.842216 23782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:47:26.842240 23782 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:47:26.842269 23782 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41201/jwks.json
I20260812 14:47:26.843649 23782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:35359
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:26.843936 23782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:26.844238 23782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:26.849478 23795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.849472 23792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:26.849481 23791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:26.849725 23782 server_base.cc:1034] running on GCE node
I20260812 14:47:26.849979 23782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:26.850299 23782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:26.851475 23782 hybrid_clock.cc:648] HybridClock initialized: now 1786546046851458 us; error 32 us; skew 500 ppm
I20260812 14:47:26.852730 23782 webserver.cc:460] Webserver started at http://127.13.146.190:46413/ using document root <none> and password file <none>
I20260812 14:47:26.852975 23782 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:26.853045 23782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:26.853276 23782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:26.854398 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "a019a4a0066e4a9982101f9e8c588676"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "d4439fe295677cabb0c46f88f9ae5ac5"
server_key_iv: "f28cccef9f73d70d3898e7e819b3917e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.854786 23782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "a019a4a0066e4a9982101f9e8c588676"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "d4439fe295677cabb0c46f88f9ae5ac5"
server_key_iv: "f28cccef9f73d70d3898e7e819b3917e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.856168 23782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:26.856915 23807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:26.857131 23782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:26.857259 23782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a019a4a0066e4a9982101f9e8c588676"
format_stamp: "Formatted at 2026-08-12 14:47:26 on dist-test-slave-8hhm"
server_key: "d4439fe295677cabb0c46f88f9ae5ac5"
server_key_iv: "f28cccef9f73d70d3898e7e819b3917e"
server_key_version: "encryptionkey@0"
I20260812 14:47:26.857350 23782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:26.900373 23782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:26.900738 23782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:26.900899 23782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:26.905341 23782 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:35359
I20260812 14:47:26.905411 23900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:35359 every 8 connection(s)
I20260812 14:47:26.905774 23782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:26.906461 23902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:26.909531 23902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676: Bootstrap starting.
I20260812 14:47:26.910176 23902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:26.910434 23902 log.cc:824] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:26.911158 23902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676: No bootstrap required, opened a new log
I20260812 14:47:26.912124 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23782
I20260812 14:47:26.912242 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:26.912422 13898 external_mini_cluster.cc:1428] Setting key fe69b5c8bf4d56819aee45a2d38470ef
I20260812 14:47:26.913192 23902 raft_consensus.cc:348] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } }
I20260812 14:47:26.913400 23902 raft_consensus.cc:374] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:26.913440 23902 raft_consensus.cc:729] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a019a4a0066e4a9982101f9e8c588676, State: Initialized, Role: FOLLOWER
I20260812 14:47:26.913578 23902 consensus_queue.cc:260] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } }
I20260812 14:47:26.913668 23902 raft_consensus.cc:388] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:26.913702 23902 raft_consensus.cc:482] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:26.913751 23902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:26.914497 23902 raft_consensus.cc:504] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } }
I20260812 14:47:26.914616 23902 leader_election.cc:302] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a019a4a0066e4a9982101f9e8c588676; no voters:
I20260812 14:47:26.914775 23902 leader_election.cc:288] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:26.914953 23909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:26.915019 23902 sys_catalog.cc:564] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:26.915164 23909 raft_consensus.cc:686] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [term 1 LEADER]: Becoming Leader. State: Replica: a019a4a0066e4a9982101f9e8c588676, State: Running, Role: LEADER
I20260812 14:47:26.915266 23909 consensus_queue.cc:237] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } }
I20260812 14:47:26.915630 23909 sys_catalog.cc:455] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a019a4a0066e4a9982101f9e8c588676. Latest consensus state: current_term: 1 leader_uuid: "a019a4a0066e4a9982101f9e8c588676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } } }
I20260812 14:47:26.915724 23909 sys_catalog.cc:458] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.915643 23920 sys_catalog.cc:455] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a019a4a0066e4a9982101f9e8c588676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a019a4a0066e4a9982101f9e8c588676" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35359 } } }
I20260812 14:47:26.915831 23920 sys_catalog.cc:458] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:26.916152 23926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:26.916798 23926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:26.918507 23926 catalog_manager.cc:1347] Generated new cluster ID: c0c25bf026c64ec29cfcb1c090c94da7
I20260812 14:47:26.918578 23926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:26.919396 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.912987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.13.146.190:35359)
0812 14:47:26.913292 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.913301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.913335 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:47:26.913451 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.913459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.913540 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.913698 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:26.913908 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.916061 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.917255 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.917806 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.917933 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.918670 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:26.918674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:26.918675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:26.918877 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:26.918915 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:26.918928 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:26.918992 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:26.919120 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 14:47:26.919194 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":124,"threads_started":1}
I20260812 14:47:26.920372 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:47:26.949704 23926 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:26.950497 23926 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:26.957540 23926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a019a4a0066e4a9982101f9e8c588676: Generated new TSK 0
I20260812 14:47:26.957734 23926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:26.970448 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:26.966259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.13.146.190:35359)
0812 14:47:26.966322 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:26.966324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:26.966434 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:47:26.966479 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:26.966480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:26.966492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:26.966590 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:47:26.966702 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.967481 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.968283 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.968605 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.968679 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.968836 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:47:26.970228 (+ 1392us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:47:26.970372 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 14:47:26.970523 23908 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 14:47:26.972396 23948 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:26.966162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35359 (local address 127.0.0.1:59246)
0812 14:47:26.966352 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:26.966361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:26.966431 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:26.966585 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:26.966585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:26.966587 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:47:26.966644 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.966645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.967589 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.967590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.968197 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:26.968199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:26.968690 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:26.968691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:26.968731 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:26.968736 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:47:26.970347 (+ 1611us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:47:26.972294 (+ 1947us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.146.190:35359: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1740,"thread_start_us":123,"threads_started":1}
I20260812 14:47:26.972746 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23782
E20260812 14:47:26.986223 23952 webserver.cc:533] Webserver: sslize failed:
E20260812 14:47:26.986301 23952 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T14:47:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (269 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T14:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:27Z Disabled control of system clock
I20260812 14:47:27.025087 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44289
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:44289
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/testchainca.cert with env {}
W20260812 14:47:27.107429 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:27.107650 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:27.107674 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:27.109019 23956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:27.109066 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:27.109078 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:27.109092 23956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:27.109104 23956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:47:27.109123 23956 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:47:27.109135 23956 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 14:47:27.110795 23956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:44289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:27.111059 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:27.111302 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:27.116474 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:27.116469 23975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:27.116503 23971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:27.116780 23956 server_base.cc:1034] running on GCE node
I20260812 14:47:27.117072 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:27.117415 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:27.118605 23956 hybrid_clock.cc:648] HybridClock initialized: now 1786546047118588 us; error 47 us; skew 500 ppm
I20260812 14:47:27.119966 23956 webserver.cc:460] Webserver started at http://127.13.146.190:41351/ using document root <none> and password file <none>
I20260812 14:47:27.120229 23956 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:27.120304 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:27.120469 23956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:27.121711 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "9a3490893e5646b3b781ca72c7b1787f"
format_stamp: "Formatted at 2026-08-12 14:47:27 on dist-test-slave-8hhm"
server_key: "1b4cb712ed892b01a37a4b4f6086b888"
server_key_iv: "d8130e7fe50af36e3bd604ecc0764441"
server_key_version: "encryptionkey@0"
I20260812 14:47:27.122118 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "9a3490893e5646b3b781ca72c7b1787f"
format_stamp: "Formatted at 2026-08-12 14:47:27 on dist-test-slave-8hhm"
server_key: "1b4cb712ed892b01a37a4b4f6086b888"
server_key_iv: "d8130e7fe50af36e3bd604ecc0764441"
server_key_version: "encryptionkey@0"
I20260812 14:47:27.123657 23956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:27.124486 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:27.124706 23956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:27.124830 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "9a3490893e5646b3b781ca72c7b1787f"
format_stamp: "Formatted at 2026-08-12 14:47:27 on dist-test-slave-8hhm"
server_key: "1b4cb712ed892b01a37a4b4f6086b888"
server_key_iv: "d8130e7fe50af36e3bd604ecc0764441"
server_key_version: "encryptionkey@0"
I20260812 14:47:27.124914 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:27.135196 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:27.135581 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:27.135752 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:27.140228 23956 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:44289
I20260812 14:47:27.140265 24082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:44289 every 8 connection(s)
I20260812 14:47:27.140671 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:27.141822 24083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:27.144199 24083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f: Bootstrap starting.
I20260812 14:47:27.144829 24083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:27.145141 24083 log.cc:824] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:27.145885 24083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f: No bootstrap required, opened a new log
I20260812 14:47:27.147533 24083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } }
I20260812 14:47:27.147677 24083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:27.147701 24083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a3490893e5646b3b781ca72c7b1787f, State: Initialized, Role: FOLLOWER
I20260812 14:47:27.147900 24083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } }
I20260812 14:47:27.147992 24083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:27.148017 24083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:27.148046 24083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:27.148572 24083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } }
I20260812 14:47:27.148666 24083 leader_election.cc:302] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a3490893e5646b3b781ca72c7b1787f; no voters:
I20260812 14:47:27.148861 24083 leader_election.cc:288] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:27.148979 24090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:27.149163 24083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:27.149194 24090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [term 1 LEADER]: Becoming Leader. State: Replica: 9a3490893e5646b3b781ca72c7b1787f, State: Running, Role: LEADER
I20260812 14:47:27.149344 24090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } }
I20260812 14:47:27.149755 24094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a3490893e5646b3b781ca72c7b1787f. Latest consensus state: current_term: 1 leader_uuid: "9a3490893e5646b3b781ca72c7b1787f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } } }
I20260812 14:47:27.149863 24094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:27.149941 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23956
I20260812 14:47:27.149943 24092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a3490893e5646b3b781ca72c7b1787f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a3490893e5646b3b781ca72c7b1787f" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44289 } } }
I20260812 14:47:27.150067 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:27.150141 24092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:27.150290 13898 external_mini_cluster.cc:1428] Setting key 31669d38c7a3012b895061654aac92a2
I20260812 14:47:27.150444 24118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:27.151182 24118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:27.152971 24118 catalog_manager.cc:1347] Generated new cluster ID: 0d7a57bd2c6548319b9bd656d1a918ad
I20260812 14:47:27.153026 24118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:27.156591 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:27.151015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.13.146.190:44289)
0812 14:47:27.151242 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:27.151251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:27.151286 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:47:27.151433 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:27.151441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:27.151512 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:27.151666 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:27.151927 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.153169 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.154288 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.154912 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.155030 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:27.155777 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:27.155780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:27.155780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:27.155982 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:27.156031 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:27.156044 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:27.156106 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:27.156213 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 14:47:27.156386 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260812 14:47:27.157527 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:47:27.162289 24118 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:27.162993 24118 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:27.166458 24118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a3490893e5646b3b781ca72c7b1787f: Generated new TSK 0
I20260812 14:47:27.166647 24118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:27.230396 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:27.174260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.13.146.190:44289)
0812 14:47:27.174333 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:27.174335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:27.174474 (+ 139us) server_negotiation.cc:407] Connection header received
0812 14:47:27.174561 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:27.174562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:27.174574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:27.174662 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:47:27.174808 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.175285 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.176119 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.176361 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.176438 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:27.176637 (+ 199us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:47:27.230098 (+ 53461us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:47:27.230308 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59676: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 14:47:27.230490 24145 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59676: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 14:47:27.230489 24160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:27.174213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44289 (local address 127.0.0.1:59676)
0812 14:47:27.174403 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:27.174412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:27.174490 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:27.174660 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:27.174661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:27.174662 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:47:27.174734 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:27.174736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.175391 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.175393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:27.176023 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:27.176025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:27.176443 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:27.176444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:27.176515 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:27.176521 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:47:27.230245 (+ 53724us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:47:27.230372 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.146.190:44289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
I20260812 14:47:27.230978 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23956
2026-08-12T14:47:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (241 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (95 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (99 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:28 dist-test-slave-8hhm krb5kdc[24178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:28 dist-test-slave-8hhm krb5kdc[24178](info): set up 2 sockets
Aug 12 14:47:28 dist-test-slave-8hhm krb5kdc[24178](info): commencing operation
krb5kdc: starting...
W20260812 14:47:30.115295 13898 mini_kdc.cc:121] Time spent starting KDC: real 2.662s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:30.242101 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37341
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:30.328464 24199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:30.328703 24199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:30.328727 24199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:30.329982 24199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:30.330034 24199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:30.330047 24199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:30.330060 24199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:30.330137 24199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:30.331674 24199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:30.331967 24199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:30.332219 24199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:30.337463 24213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.337503 24199 server_base.cc:1034] running on GCE node
W20260812 14:47:30.337469 24215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.337463 24217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.337987 24199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:30.338304 24199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:30.339480 24199 hybrid_clock.cc:648] HybridClock initialized: now 1786546050339472 us; error 37 us; skew 500 ppm
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.342142 24199 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:30.342581 24199 webserver.cc:460] Webserver started at http://127.13.146.190:46441/ using document root <none> and password file <none>
I20260812 14:47:30.342823 24199 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:30.342893 24199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:30.343050 24199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:30.344168 24199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "14519f0e101b42e1924b223e25b3221a"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "a43805b54023d60ce20d5ec0e1cb005b"
server_key_iv: "2a49efff497e0100582589743a7f6dcc"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.344554 24199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "14519f0e101b42e1924b223e25b3221a"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "a43805b54023d60ce20d5ec0e1cb005b"
server_key_iv: "2a49efff497e0100582589743a7f6dcc"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.346012 24199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:30.346767 24234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.346994 24199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:30.347107 24199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "14519f0e101b42e1924b223e25b3221a"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "a43805b54023d60ce20d5ec0e1cb005b"
server_key_iv: "2a49efff497e0100582589743a7f6dcc"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.347186 24199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:30.359386 24199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:30.361685 24199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:30.361865 24199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:30.366216 24199 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:37341
I20260812 14:47:30.366262 24337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:37341 every 8 connection(s)
I20260812 14:47:30.366663 24199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:30.367573 24339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:30.368176 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24199
I20260812 14:47:30.368312 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:30.368499 13898 external_mini_cluster.cc:1428] Setting key 8e122f9f6a09fc26c82774eacbe12a71
I20260812 14:47:30.370821 24339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: Bootstrap starting.
I20260812 14:47:30.371479 24339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:30.371816 24339 log.cc:824] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:30.372555 24339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: No bootstrap required, opened a new log
I20260812 14:47:30.374500 24339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } }
I20260812 14:47:30.374696 24339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:30.374735 24339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14519f0e101b42e1924b223e25b3221a, State: Initialized, Role: FOLLOWER
I20260812 14:47:30.374913 24339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } }
I20260812 14:47:30.375001 24339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:30.375038 24339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:30.375093 24339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:30.375880 24339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } }
I20260812 14:47:30.376024 24339 leader_election.cc:302] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14519f0e101b42e1924b223e25b3221a; no voters:
I20260812 14:47:30.376241 24339 leader_election.cc:288] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:30.376278 24350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:30.376433 24350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [term 1 LEADER]: Becoming Leader. State: Replica: 14519f0e101b42e1924b223e25b3221a, State: Running, Role: LEADER
I20260812 14:47:30.376531 24350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } }
I20260812 14:47:30.376572 24339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:30.377853 24354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14519f0e101b42e1924b223e25b3221a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } } }
I20260812 14:47:30.377941 24354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:30.378242 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14519f0e101b42e1924b223e25b3221a. Latest consensus state: current_term: 1 leader_uuid: "14519f0e101b42e1924b223e25b3221a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14519f0e101b42e1924b223e25b3221a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37341 } } }
I20260812 14:47:30.378448 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a [sys.catalog]: This master's current role is: LEADER
W20260812 14:47:30.378633 24378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:30.378741 24378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:30.378846 24377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:30.379040 24377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:30.380270 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.369085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.13.146.190:37341)
0812 14:47:30.369292 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.369300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.369327 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:47:30.369736 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.369743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.369790 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.369906 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.370776 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.371836 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.372629 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.373032 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.373178 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.375920 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.375924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.375924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.375972 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.378397 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.378831 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.378833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.378834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.378941 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.379207 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.379208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.379208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.379509 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.379666 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.379791 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.380046 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260812 14:47:30.380700 24377 catalog_manager.cc:1347] Generated new cluster ID: 9c2436388f804432ba8b17ef4c3ac996
I20260812 14:47:30.380750 24377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:30.388149 24377 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:30.388664 24377 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:30.391518 24377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14519f0e101b42e1924b223e25b3221a: Generated new TSK 0
I20260812 14:47:30.391697 24377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:30.428498 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:30.521348 24393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:30.521574 24393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:30.521600 24393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:30.522702 24393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:30.522738 24393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:30.522763 24393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:30.524271 24393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:30.524458 24393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:30.524650 24393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:30.529736 24409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.529767 24413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.529762 24408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.530017 24393 server_base.cc:1034] running on GCE node
I20260812 14:47:30.530232 24393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:30.530471 24393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:30.531615 24393 hybrid_clock.cc:648] HybridClock initialized: now 1786546050531602 us; error 30 us; skew 500 ppm
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.534281 24393 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:30.534721 24393 webserver.cc:460] Webserver started at http://127.13.146.129:44181/ using document root <none> and password file <none>
I20260812 14:47:30.534952 24393 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:30.534998 24393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:30.535166 24393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:30.536257 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "4916d4305ede4dd2a7c9f41e9b9c35f8"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "3daba920f3273810771568c8e73f548a"
server_key_iv: "4e504d68e09ed6b2fb1e5cff8ee9edbd"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.536643 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "4916d4305ede4dd2a7c9f41e9b9c35f8"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "3daba920f3273810771568c8e73f548a"
server_key_iv: "4e504d68e09ed6b2fb1e5cff8ee9edbd"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.538108 24393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:30.538865 24429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.539067 24393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:30.539181 24393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "4916d4305ede4dd2a7c9f41e9b9c35f8"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "3daba920f3273810771568c8e73f548a"
server_key_iv: "4e504d68e09ed6b2fb1e5cff8ee9edbd"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.539258 24393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:30.546098 24393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:30.548053 24393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:30.548218 24393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:30.548466 24393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:30.548779 24393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:30.548836 24393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.548899 24393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:30.548945 24393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.555423 24393 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:45085
I20260812 14:47:30.555519 24666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:45085 every 8 connection(s)
I20260812 14:47:30.555822 24393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:30.564853 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24393
I20260812 14:47:30.565001 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:30.565270 13898 external_mini_cluster.cc:1428] Setting key 1781830ad90d123a5d3f42e2cd157ea0
I20260812 14:47:30.565403 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.556604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:33565 (local address 127.13.146.190:37341)
0812 14:47:30.556676 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.556678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.556854 (+ 176us) server_negotiation.cc:407] Connection header received
0812 14:47:30.557893 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.557894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.557904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.557967 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.558638 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.559147 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.560131 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.560335 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.560388 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.562560 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.562561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.562561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.562568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.563872 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.564271 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.564272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.564272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.564325 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.564604 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.564604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.564604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.564778 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.564859 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.565191 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.565307 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:47:30.565701 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.556556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37341 (local address 127.13.146.129:33565)
0812 14:47:30.556723 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:30.556767 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:30.557782 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:30.558051 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:30.558054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:30.558315 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:30.558497 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.558501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.559233 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.559234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.560051 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.560053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.560410 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.560411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.560514 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.561002 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:30.561010 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:30.562471 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:30.563951 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.563958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.563960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.564202 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.564404 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.564405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.564406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.564538 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.564858 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:30.564864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:30.565019 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 14:47:30.565164 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 14:47:30.565382 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260812 14:47:30.566306 24668 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37341
I20260812 14:47:30.566407 24668 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:30.566599 24668 heartbeater.cc:511] Master 127.13.146.190:37341 requested a full tablet report, sending...
I20260812 14:47:30.567116 24273 ts_manager.cc:194] Registered new tserver with Master: 4916d4305ede4dd2a7c9f41e9b9c35f8 (127.13.146.129:45085)
I20260812 14:47:30.567750 24273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:33565
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:30.592095 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:30.675601 24681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:30.675865 24681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:30.675907 24681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:30.677176 24681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:30.677253 24681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:30.677301 24681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:30.678843 24681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:30.679121 24681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:30.679378 24681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:30.684648 24698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.684664 24696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.684648 24695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.684911 24681 server_base.cc:1034] running on GCE node
I20260812 14:47:30.685111 24681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:30.685384 24681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:30.686527 24681 hybrid_clock.cc:648] HybridClock initialized: now 1786546050686517 us; error 27 us; skew 500 ppm
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.689096 24681 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:30.689575 24681 webserver.cc:460] Webserver started at http://127.13.146.130:42821/ using document root <none> and password file <none>
I20260812 14:47:30.689807 24681 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:30.689880 24681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:30.690038 24681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:30.691103 24681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "6cf3b30d898c4bb8bde05223b39ddb24"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "112c5cde0b580838759f4d3352823def"
server_key_iv: "bea855409b5da6445110f11ee9e10422"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.691475 24681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "6cf3b30d898c4bb8bde05223b39ddb24"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "112c5cde0b580838759f4d3352823def"
server_key_iv: "bea855409b5da6445110f11ee9e10422"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.692853 24681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:30.693724 24712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.693898 24681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.694010 24681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "6cf3b30d898c4bb8bde05223b39ddb24"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "112c5cde0b580838759f4d3352823def"
server_key_iv: "bea855409b5da6445110f11ee9e10422"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.694085 24681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:30.706457 24681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:30.708379 24681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:30.708549 24681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:30.708799 24681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:30.709110 24681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:30.709164 24681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.709200 24681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:30.709246 24681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.715234 24681 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:46407
I20260812 14:47:30.715312 24970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:46407 every 8 connection(s)
I20260812 14:47:30.715637 24681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:30.717278 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24681
I20260812 14:47:30.717378 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:30.717597 13898 external_mini_cluster.cc:1428] Setting key 3b0676f4217222125fb5671978a817c5
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:30.726763 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.716415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37341 (local address 127.13.146.130:42371)
0812 14:47:30.716611 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:30.716648 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:30.717978 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:30.718186 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:30.718189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:30.718513 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:30.718791 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.718797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.719486 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.719487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.720648 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.720651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.721043 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.721044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.721178 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.721870 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:30.721885 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:30.723289 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:30.725216 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.725250 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.725252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.725613 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.725845 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.725846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.725846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.725949 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.726266 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:30.726272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:30.726393 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 14:47:30.726473 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:47:30.726592 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260812 14:47:30.726768 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.716432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42371 (local address 127.13.146.190:37341)
0812 14:47:30.716501 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.716504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.716733 (+ 229us) server_negotiation.cc:407] Connection header received
0812 14:47:30.718057 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.718058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.718070 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.718129 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.718880 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.719371 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.720757 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.720958 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.721036 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.723383 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.723384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.723385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.723393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.725097 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.725701 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.725702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.725702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.725772 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.726028 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.726030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.726030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.726190 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.726263 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.726550 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.726643 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:47:30.727283 24972 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37341
I20260812 14:47:30.727387 24972 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:30.727587 24972 heartbeater.cc:511] Master 127.13.146.190:37341 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:47:30.728022 24273 ts_manager.cc:194] Registered new tserver with Master: 6cf3b30d898c4bb8bde05223b39ddb24 (127.13.146.130:46407)
I20260812 14:47:30.728400 24273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:42371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:30.747395 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:30.831578 24984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:30.831810 24984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:30.831835 24984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:30.833103 24984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:30.833153 24984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:30.833179 24984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:30.834707 24984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:30.834905 24984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:30.835093 24984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:30.840178 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.840240 24984 server_base.cc:1034] running on GCE node
W20260812 14:47:30.840193 24998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:30.840198 25000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:30.840662 24984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:30.840907 24984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:30.842108 24984 hybrid_clock.cc:648] HybridClock initialized: now 1786546050842098 us; error 49 us; skew 500 ppm
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.844726 24984 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:30.845280 24984 webserver.cc:460] Webserver started at http://127.13.146.131:39837/ using document root <none> and password file <none>
I20260812 14:47:30.845541 24984 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:30.845611 24984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:30.845767 24984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:30.846802 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "5e3dc5c9767e4d74947810eeb893c8b9"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "5d03ab8cb5985d37512b05dba97a06a1"
server_key_iv: "1ed12daf7c32143b919288a7c0886d89"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.847164 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "5e3dc5c9767e4d74947810eeb893c8b9"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "5d03ab8cb5985d37512b05dba97a06a1"
server_key_iv: "1ed12daf7c32143b919288a7c0886d89"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.848464 24984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:30.849179 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.849426 24984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:30.849517 24984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "5e3dc5c9767e4d74947810eeb893c8b9"
format_stamp: "Formatted at 2026-08-12 14:47:30 on dist-test-slave-8hhm"
server_key: "5d03ab8cb5985d37512b05dba97a06a1"
server_key_iv: "1ed12daf7c32143b919288a7c0886d89"
server_key_version: "encryptionkey@0"
I20260812 14:47:30.849573 24984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:30.868498 24984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:30.871147 24984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:30.871328 24984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:30.871584 24984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:30.871933 24984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:30.871991 24984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.872027 24984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:30.872054 24984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:30.878096 24984 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:32781
I20260812 14:47:30.878435 25260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:32781 every 8 connection(s)
I20260812 14:47:30.878573 24984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:30.889402 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24984
I20260812 14:47:30.889494 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:30.889678 13898 external_mini_cluster.cc:1428] Setting key 772981a69fb2771d7b012ff183502c8b
I20260812 14:47:30.890147 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.879250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:60517 (local address 127.13.146.190:37341)
0812 14:47:30.879318 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.879320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.879558 (+ 238us) server_negotiation.cc:407] Connection header received
0812 14:47:30.880727 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.880728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.880741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.880808 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.881669 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.882394 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.883788 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.884057 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.884096 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.886762 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.886763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.886764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.886772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.888395 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.888883 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.888884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.888885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.888948 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.889262 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.889264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.889264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.889450 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.889537 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.889918 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.890017 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:47:30.890295 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.879258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37341 (local address 127.13.146.131:60517)
0812 14:47:30.879434 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:30.879470 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:30.880599 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:30.880882 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:30.880886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:30.881273 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:30.881540 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.881547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.882510 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.882511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.883687 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:30.883689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.884183 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.884183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:30.884323 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.884889 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:30.884899 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:30.886649 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:30.888493 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.888500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.888503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.888795 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.889019 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:30.889019 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:30.889020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:30.889158 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:30.889572 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:30.889579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:30.889750 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 14:47:30.889869 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 14:47:30.890060 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260812 14:47:30.890772 25262 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37341
I20260812 14:47:30.890892 25262 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:30.891079 25262 heartbeater.cc:511] Master 127.13.146.190:37341 requested a full tablet report, sending...
I20260812 14:47:30.891484 24273 ts_manager.cc:194] Registered new tserver with Master: 5e3dc5c9767e4d74947810eeb893c8b9 (127.13.146.131:32781)
I20260812 14:47:30.892077 24273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:60517
I20260812 14:47:30.892961 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/test-admin.keytab.
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.910194 13898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 14:47:30.912075 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:30.911227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.13.146.129:45085)
0812 14:47:30.911443 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.911447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.911458 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:30.911487 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.911489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.911504 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.911573 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.912010 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35532: BlockingRecv error: recv got EOF from 127.0.0.1:35532 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
W20260812 14:47:30.912336 13898 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.913444 13898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.914372 13898 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:30.921977 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.914922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.13.146.129:45085)
0812 14:47:30.914988 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.914990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.914999 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:30.915039 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.915040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.915049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.915096 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.915488 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.916210 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.916888 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.917194 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.917305 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.919434 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.919442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.919442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.919449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.920828 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.921146 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.921149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.921149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.921208 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.921452 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.921453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.921453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.921586 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.921710 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.921762 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.921822 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 14:47:30.923811 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:30.923216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.13.146.129:45085)
0812 14:47:30.923280 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.923282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.923290 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:30.923380 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.923381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.923398 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.923461 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.923749 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35550: BlockingRecv error: recv got EOF from 127.0.0.1:35550 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 14:47:30.923929 13898 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.924855 13898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.925758 13898 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:30.932157 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.926254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.13.146.129:45085)
0812 14:47:30.926310 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.926312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.926369 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:47:30.926435 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.926436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.926444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.926488 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.926810 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.927258 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.927899 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.928097 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.928144 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.930016 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.930017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.930017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.930023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.931168 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.931463 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.931464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.931464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.931512 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.931702 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.931703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.931703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.931793 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.931861 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.931958 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.932014 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 14:47:30.933594 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:30.933078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.13.146.129:45085)
0812 14:47:30.933113 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.933114 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.933154 (+ 40us) server_negotiation.cc:407] Connection header received
0812 14:47:30.933203 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.933204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.933258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.933311 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.933555 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35568: BlockingRecv error: recv got EOF from 127.0.0.1:35568 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:47:30.933681 13898 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786546020513802-13898-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.934561 13898 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:30.935382 13898 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:47:30 dist-test-slave-8hhm krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:47:30.942333 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:30.935868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.13.146.129:45085)
0812 14:47:30.935932 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:30.935934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:30.935997 (+ 63us) server_negotiation.cc:407] Connection header received
0812 14:47:30.936081 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:30.936082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:30.936091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:30.936144 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:30.936497 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.936970 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.937735 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:30.937936 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:30.938021 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:30.939964 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:30.939965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:30.939966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:30.939973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:30.941163 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.941565 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.941566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.941567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.941630 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:30.941842 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:30.941843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:30.941844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:30.941944 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:30.942064 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:30.942154 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 14:47:30.942231 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:30.943141 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24393
I20260812 14:47:30.961735 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24681
I20260812 14:47:30.978153 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24984
I20260812 14:47:30.992126 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24199
2026-08-12T14:47:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3568 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:31 dist-test-slave-8hhm krb5kdc[25311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:31 dist-test-slave-8hhm krb5kdc[25311](info): set up 2 sockets
Aug 12 14:47:31 dist-test-slave-8hhm krb5kdc[25311](info): commencing operation
krb5kdc: starting...
W20260812 14:47:32.332499 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab.
Entry for principal oryx/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab.
I20260812 14:47:32.459816 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34779
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab
--principal=oryx/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:32.542876 25334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:32.543090 25334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:32.543112 25334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:32.544358 25334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:32.544407 25334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:32.544421 25334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:32.544435 25334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:32.544486 25334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:32.546065 25334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab
--principal=oryx/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:32.546334 25334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:32.546588 25334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:32.551807 25348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:32.551829 25345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:32.551949 25334 server_base.cc:1034] running on GCE node
W20260812 14:47:32.551883 25343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:32.552366 25334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:32.552682 25334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:32.553879 25334 hybrid_clock.cc:648] HybridClock initialized: now 1786546052553856 us; error 39 us; skew 500 ppm
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:32.556646 25334 init.cc:377] Logged in from keytab as oryx/127.13.146.190@KRBTEST.COM (short username oryx)
I20260812 14:47:32.557140 25334 webserver.cc:460] Webserver started at http://127.13.146.190:41657/ using document root <none> and password file <none>
I20260812 14:47:32.557396 25334 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:32.557444 25334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:32.557597 25334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:32.558703 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "503ebdeceae3488bbd9bff48396e5e07"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "c7d48b619627ef8caee169c1cb54a4e1"
server_key_iv: "8cde414e838df4d81e78a6f273b3592b"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.559074 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "503ebdeceae3488bbd9bff48396e5e07"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "c7d48b619627ef8caee169c1cb54a4e1"
server_key_iv: "8cde414e838df4d81e78a6f273b3592b"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.560429 25334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:32.561200 25366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.561429 25334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:32.561510 25334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "503ebdeceae3488bbd9bff48396e5e07"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "c7d48b619627ef8caee169c1cb54a4e1"
server_key_iv: "8cde414e838df4d81e78a6f273b3592b"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.561600 25334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:32.576493 25334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:32.578960 25334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:32.579082 25334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:32.583392 25334 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34779
I20260812 14:47:32.583436 25476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34779 every 8 connection(s)
I20260812 14:47:32.583823 25334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:32.584903 25477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:32.585400 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 25334
I20260812 14:47:32.585489 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:32.585706 13898 external_mini_cluster.cc:1428] Setting key edfea14bbc0dc5a684cb43ebe17e8ecb
I20260812 14:47:32.587680 25477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Bootstrap starting.
I20260812 14:47:32.588320 25477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:32.588631 25477 log.cc:824] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:32.589414 25477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: No bootstrap required, opened a new log
I20260812 14:47:32.591468 25477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } }
I20260812 14:47:32.591668 25477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:32.591714 25477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503ebdeceae3488bbd9bff48396e5e07, State: Initialized, Role: FOLLOWER
I20260812 14:47:32.591853 25477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } }
I20260812 14:47:32.591958 25477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:32.591993 25477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:32.592043 25477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:32.592775 25477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } }
I20260812 14:47:32.592891 25477 leader_election.cc:302] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 503ebdeceae3488bbd9bff48396e5e07; no voters:
I20260812 14:47:32.593104 25477 leader_election.cc:288] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:32.593132 25494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:32.593407 25477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:32.593583 25494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 LEADER]: Becoming Leader. State: Replica: 503ebdeceae3488bbd9bff48396e5e07, State: Running, Role: LEADER
I20260812 14:47:32.593752 25494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } }
I20260812 14:47:32.594485 25497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 503ebdeceae3488bbd9bff48396e5e07. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } }
I20260812 14:47:32.594596 25497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:32.595028 25495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } }
W20260812 14:47:32.595309 25529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:32.595729 25529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:32.595805 25523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:32.595997 25523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:32.596900 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:32.586274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.13.146.190:34779)
0812 14:47:32.586461 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:32.586469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:32.586495 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:47:32.586914 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:32.586921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:32.586968 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:32.587112 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:32.587755 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.588650 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.589505 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.589939 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.590053 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:32.592882 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:32.592887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:32.592887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:32.592902 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:32.595084 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:32.595562 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:32.595566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:32.595566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:32.595698 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:32.595930 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:32.595931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:32.595932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:32.596256 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:32.596410 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:32.596545 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 14:47:32.596708 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260812 14:47:32.597000 25495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:32.597663 25523 catalog_manager.cc:1347] Generated new cluster ID: 297848a9d8cb472eaffcaf4f4d9d2bf1
I20260812 14:47:32.597712 25523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:32.647361 25523 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:32.648736 25523 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:32.655277 25523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Generated new TSK 0
I20260812 14:47:32.655494 25523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
I20260812 14:47:32.686594 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:32.772869 25547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:32.773200 25547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:32.773314 25547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:32.774523 25547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:32.774564 25547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:32.774590 25547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:32.776008 25547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:32.776212 25547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:32.776386 25547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:32.781672 25563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:32.781641 25558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:32.781731 25547 server_base.cc:1034] running on GCE node
W20260812 14:47:32.781665 25560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:32.782099 25547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:32.782371 25547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:32.783556 25547 hybrid_clock.cc:648] HybridClock initialized: now 1786546052783538 us; error 34 us; skew 500 ppm
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:32.786278 25547 init.cc:377] Logged in from keytab as oryx/127.13.146.129@KRBTEST.COM (short username oryx)
I20260812 14:47:32.786684 25547 webserver.cc:460] Webserver started at http://127.13.146.129:36851/ using document root <none> and password file <none>
I20260812 14:47:32.786867 25547 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:32.786907 25547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:32.786993 25547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:32.788031 25547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "cac29c3e72f146a1b79655e2a9fa52e3"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "360b1a5a68037c54357c19fa1b4037c7"
server_key_iv: "ef8ac1f1b2bddda9c0c654b6a86450b5"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.788355 25547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "cac29c3e72f146a1b79655e2a9fa52e3"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "360b1a5a68037c54357c19fa1b4037c7"
server_key_iv: "ef8ac1f1b2bddda9c0c654b6a86450b5"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.789788 25547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:32.790570 25575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.790766 25547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:32.790885 25547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "cac29c3e72f146a1b79655e2a9fa52e3"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "360b1a5a68037c54357c19fa1b4037c7"
server_key_iv: "ef8ac1f1b2bddda9c0c654b6a86450b5"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.790968 25547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:32.826786 25547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:32.829352 25547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:32.829530 25547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:32.829845 25547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:32.830192 25547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:32.830252 25547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.830299 25547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:32.830327 25547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.836521 25547 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44681
I20260812 14:47:32.836573 25808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44681 every 8 connection(s)
I20260812 14:47:32.836915 25547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:32.842943 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 25547
I20260812 14:47:32.843055 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:32.843255 13898 external_mini_cluster.cc:1428] Setting key 1c2130704229567e1f5633d0316a1ded
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:32.847142 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:32.837707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:33449)
0812 14:47:32.837891 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:32.837935 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:32.838448 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:32.838668 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:32.838671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:32.839253 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:32.839450 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:32.839456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.840185 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.840186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:32.841017 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:32.841020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.841464 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.841464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:32.841573 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:32.842029 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:32.842040 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:32.843756 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:32.845608 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:32.845617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:32.845619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:32.845964 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:32.846215 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:32.846216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:32.846216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:32.846328 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:32.846637 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:32.846642 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:32.846758 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 14:47:32.846840 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:47:32.846963 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260812 14:47:32.847136 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:32.837725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:33449 (local address 127.13.146.190:34779)
0812 14:47:32.837788 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:32.837791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:32.838023 (+ 232us) server_negotiation.cc:407] Connection header received
0812 14:47:32.838533 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:32.838534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:32.838543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:32.838600 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:32.839533 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.840049 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.841102 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:32.841389 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:32.841447 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:32.843859 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:32.843861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:32.843861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:32.843869 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:32.845522 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:32.846049 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:32.846051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:32.846051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:32.846135 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:32.846400 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:32.846401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:32.846401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:32.846563 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:32.846646 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:32.846892 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 14:47:32.847012 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:47:32.847733 25809 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:32.847851 25809 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:32.848079 25809 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:32.848657 25410 ts_manager.cc:194] Registered new tserver with Master: cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129:44681)
I20260812 14:47:32.849406 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:33449
WARNING: no policy specified for oryx/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
I20260812 14:47:32.870237 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:32.954599 25822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:32.954847 25822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:32.954886 25822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:32.956028 25822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:32.956108 25822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:32.956154 25822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:32.957701 25822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:32.957991 25822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:32.958244 25822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:32.963593 25836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:32.963617 25830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:32.963639 25831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:32.963707 25822 server_base.cc:1034] running on GCE node
I20260812 14:47:32.964089 25822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:32.964417 25822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:32.965581 25822 hybrid_clock.cc:648] HybridClock initialized: now 1786546052965568 us; error 40 us; skew 500 ppm
Aug 12 14:47:32 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:32.968245 25822 init.cc:377] Logged in from keytab as oryx/127.13.146.130@KRBTEST.COM (short username oryx)
I20260812 14:47:32.968763 25822 webserver.cc:460] Webserver started at http://127.13.146.130:40905/ using document root <none> and password file <none>
I20260812 14:47:32.969030 25822 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:32.969103 25822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:32.969296 25822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:32.970387 25822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "a2b0bb258c2d4227a97db8d327158d6b"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "62ffba8fa0e3ade77c16dd5e9d7c1f38"
server_key_iv: "b30d98fffa7b656317bc4358b7e4b031"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.970767 25822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "a2b0bb258c2d4227a97db8d327158d6b"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "62ffba8fa0e3ade77c16dd5e9d7c1f38"
server_key_iv: "b30d98fffa7b656317bc4358b7e4b031"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.972138 25822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:32.972860 25848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.973043 25822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:32.973155 25822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "a2b0bb258c2d4227a97db8d327158d6b"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "62ffba8fa0e3ade77c16dd5e9d7c1f38"
server_key_iv: "b30d98fffa7b656317bc4358b7e4b031"
server_key_version: "encryptionkey@0"
I20260812 14:47:32.973263 25822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:32.991168 25822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:32.993184 25822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:32.993412 25822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:32.993652 25822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:32.994071 25822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:32.994131 25822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:32.994172 25822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:32.994194 25822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.000440 25822 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34839
I20260812 14:47:33.000494 26091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34839 every 8 connection(s)
I20260812 14:47:33.000825 25822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:33.005657 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 25822
I20260812 14:47:33.005749 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:33.005956 13898 external_mini_cluster.cc:1428] Setting key 48d590a58ac987cd563cf774b7563512
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:33.010886 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.001584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:42435)
0812 14:47:33.001801 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.001844 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.002341 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.002559 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.002562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.003160 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.003360 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.003365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.004022 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.004023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.004857 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.004860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.005255 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.005256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.005385 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.006072 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.006083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.007675 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.009473 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.009481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.009484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.009719 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.009945 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.009946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.009946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.010060 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.010368 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.010374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.010497 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.010587 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.010717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":132,"threads_started":1}
I20260812 14:47:33.010888 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.001626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42435 (local address 127.13.146.190:34779)
0812 14:47:33.001700 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.001702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.001932 (+ 230us) server_negotiation.cc:407] Connection header received
0812 14:47:33.002422 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.002423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.002433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.002489 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.003445 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.003933 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.004940 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.005125 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.005181 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.007781 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.007783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.007784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.007791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.009379 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.009801 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.009802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.009802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.009869 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.010136 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.010137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.010137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.010291 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.010364 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.010641 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.010747 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:47:33.011416 26092 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:33.011529 26092 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:33.011739 26092 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:33.012207 25410 ts_manager.cc:194] Registered new tserver with Master: a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839)
I20260812 14:47:33.012765 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:42435
WARNING: no policy specified for oryx/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
I20260812 14:47:33.034443 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:33.118717 26101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:33.118917 26101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:33.118942 26101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:33.120132 26101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:33.120177 26101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:33.120203 26101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:33.121730 26101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:33.121968 26101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:33.122200 26101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.127465 26117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.127503 26113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.127517 26112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.127735 26101 server_base.cc:1034] running on GCE node
I20260812 14:47:33.127916 26101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.128116 26101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.129276 26101 hybrid_clock.cc:648] HybridClock initialized: now 1786546053129262 us; error 33 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.131975 26101 init.cc:377] Logged in from keytab as oryx/127.13.146.131@KRBTEST.COM (short username oryx)
I20260812 14:47:33.132414 26101 webserver.cc:460] Webserver started at http://127.13.146.131:38987/ using document root <none> and password file <none>
I20260812 14:47:33.132640 26101 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.132685 26101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.132840 26101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:33.133957 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "c1372e01bfde4a21823e90d1412d6539"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "82ec26a93b41d92c3ace84d3f4572d4d"
server_key_iv: "cc60a716bc12b0f7c765d8e45a8318aa"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.134341 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "c1372e01bfde4a21823e90d1412d6539"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "82ec26a93b41d92c3ace84d3f4572d4d"
server_key_iv: "cc60a716bc12b0f7c765d8e45a8318aa"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.135718 26101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:33.136533 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.136793 26101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:33.136929 26101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "c1372e01bfde4a21823e90d1412d6539"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "82ec26a93b41d92c3ace84d3f4572d4d"
server_key_iv: "cc60a716bc12b0f7c765d8e45a8318aa"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.137054 26101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.154170 26101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.156719 26101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:33.156894 26101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.157150 26101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:33.157531 26101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:33.157593 26101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.157644 26101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:33.157691 26101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.164273 26101 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36391
I20260812 14:47:33.164304 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36391 every 8 connection(s)
I20260812 14:47:33.164664 26101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:33.170408 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 26101
I20260812 14:47:33.170526 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:33.170725 13898 external_mini_cluster.cc:1428] Setting key a8c60c83116bf30610e4aef9de7d0767
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:33.174336 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.165457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:33607 (local address 127.13.146.190:34779)
0812 14:47:33.165532 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.165534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.165734 (+ 200us) server_negotiation.cc:407] Connection header received
0812 14:47:33.166257 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.166258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.166269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.166322 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.167274 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.167759 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.168774 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.168967 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.169025 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.171669 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.171670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.171671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.171679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.173064 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.173527 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.173527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.173528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.173585 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.173809 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.173810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.173810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.173916 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.173969 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.174206 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.174272 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:33.174551 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.165452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:33607)
0812 14:47:33.165623 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.165661 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.166166 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.166378 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.166382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.166991 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.167192 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.167198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.167844 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.167845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.168679 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.168682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.169209 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.169211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.169355 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.169856 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.169865 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.171576 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.173158 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.173164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.173166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.173456 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.173648 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.173648 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.173648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.173751 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.173971 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.173975 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.174090 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.174176 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.174298 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260812 14:47:33.175072 26361 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:33.175163 26361 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:33.175305 26361 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:33.175696 25410 ts_manager.cc:194] Registered new tserver with Master: c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:33.176097 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:33607
I20260812 14:47:33.182746 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.189.keytab.
Entry for principal oryx/127.13.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.189.keytab.
I20260812 14:47:33.212632 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:33909
--webserver_interface=127.13.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.189.keytab
--principal=oryx/127.13.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34779,127.13.146.189:33909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:33.297777 26367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:33.298007 26367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:33.298044 26367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:33.299201 26367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:33.299268 26367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:33.299291 26367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:33.299322 26367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:33.299350 26367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:33.300825 26367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34779,127.13.146.189:33909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.189.keytab
--principal=oryx/127.13.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
W20260812 14:47:33.300968 26367 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.146.190:34779,127.13.146.189:33909'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 14:47:33.301126 26367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:33.301375 26367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.306339 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.306401 26384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.306363 26378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.306836 26367 server_base.cc:1034] running on GCE node
I20260812 14:47:33.307111 26367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.307423 26367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.308565 26367 hybrid_clock.cc:648] HybridClock initialized: now 1786546053308543 us; error 46 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.317204 26367 init.cc:377] Logged in from keytab as oryx/127.13.146.189@KRBTEST.COM (short username oryx)
I20260812 14:47:33.317737 26367 webserver.cc:460] Webserver started at http://127.13.146.189:38755/ using document root <none> and password file <none>
I20260812 14:47:33.317996 26367 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.318066 26367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.318221 26367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:33.319381 26367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/instance:
uuid: "9dcf175f04654c318187c2467e7c9556"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "412bb65d796d9fb09c178611022bda57"
server_key_iv: "c4c45188becb7e81c6ef3ef1c9d3608c"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.319782 26367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal/instance:
uuid: "9dcf175f04654c318187c2467e7c9556"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "412bb65d796d9fb09c178611022bda57"
server_key_iv: "c4c45188becb7e81c6ef3ef1c9d3608c"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.321431 26367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:33.322345 26404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.322590 26367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:33.322702 26367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "9dcf175f04654c318187c2467e7c9556"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "412bb65d796d9fb09c178611022bda57"
server_key_iv: "c4c45188becb7e81c6ef3ef1c9d3608c"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.322769 26367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.337512 26367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.339709 26367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:33.339905 26367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.344439 26367 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.189:33909
I20260812 14:47:33.344514 26502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.189:33909 every 8 connection(s)
I20260812 14:47:33.344889 26367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
I20260812 14:47:33.346040 26503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:33.347647 26503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:47:33.348486 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 26367
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.189@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
I20260812 14:47:33.358086 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.348443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.13.146.190:34779)
0812 14:47:33.348516 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.348519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.348764 (+ 245us) server_negotiation.cc:407] Connection header received
0812 14:47:33.349537 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.349538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.349553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.349614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.350227 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.350839 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.351837 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.352064 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.352149 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.354688 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.354690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.354690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.354697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.356303 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.356932 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.356933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.356933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.356989 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.357314 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.357315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.357315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.357479 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.357537 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.357933 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.358022 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:47:33.358451 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.348387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49814)
0812 14:47:33.348611 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.348657 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.349408 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.349651 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.349656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.349950 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.350117 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.350122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.350950 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.350951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.351710 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.351712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.352165 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.352166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.352295 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.352852 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.352864 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.354599 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.356404 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.356413 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.356415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.356846 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.357067 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.357068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.357069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.357209 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.357564 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.357570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.357758 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.357894 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.358092 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
I20260812 14:47:33.359180 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.349083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.13.146.189:33909)
0812 14:47:33.349321 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.349327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.349392 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:47:33.349527 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.349529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.349546 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.349668 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.350276 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.351128 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.351884 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.352226 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.352308 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.355370 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.355377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.355377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.355384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.357498 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.357992 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.357995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.357995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.358118 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.358359 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.358360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.358360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.358533 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.358694 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.358935 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.359035 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
I20260812 14:47:33.359154 26503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.189@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
I20260812 14:47:33.365774 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.359418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53256)
0812 14:47:33.359475 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.359487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.359536 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.359672 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.359673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.359840 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.359879 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.359881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.360602 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.360603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.361135 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.361137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.361594 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.361595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.361645 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.361989 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.361993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.363252 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.364773 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.364774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.364774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.364943 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.365272 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.365276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.365277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.365334 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.365550 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.365550 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.365584 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.365642 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.365715 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:47:33.365890 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.359428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.13.146.189:33909)
0812 14:47:33.359562 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.359564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.359573 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:33.359592 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.359592 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.359602 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.359662 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.359956 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.360496 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.361293 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.361508 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.361572 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.363358 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.363359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.363360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.363366 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.364657 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.365060 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.365065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.365065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.365161 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.365391 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.365392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.365392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.365492 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.365544 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.365673 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.365762 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":117}
I20260812 14:47:33.367183 26503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Bootstrap starting.
I20260812 14:47:33.367980 26503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:33.368218 26503 log.cc:824] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:33.368963 26503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: No bootstrap required, opened a new log
I20260812 14:47:33.370461 26503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } }
I20260812 14:47:33.370587 26503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:33.370625 26503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dcf175f04654c318187c2467e7c9556, State: Initialized, Role: FOLLOWER
I20260812 14:47:33.370748 26503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } }
I20260812 14:47:33.371024 26514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } } }
I20260812 14:47:33.371097 26503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:33.371140 26514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:47:33.372958 26545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:33.373016 26545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:33.374922 26367 master_runner.cc:428] Detected that this master 9dcf175f04654c318187c2467e7c9556 is joining an existing cluster
I20260812 14:47:33.374998 26367 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.146.189:33909
I20260812 14:47:33.383200 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.376223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.13.146.190:34779)
0812 14:47:33.376271 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.376273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.376410 (+ 137us) server_negotiation.cc:407] Connection header received
0812 14:47:33.376494 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.376495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.376509 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.376571 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.377074 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.377672 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.378723 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.378965 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.378995 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.380210 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.380212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.380212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.380220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.381978 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382385 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.382386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.382387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.382444 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382667 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.382668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.382668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.382793 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.382867 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.383027 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.383107 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:33.383370 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.376490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.13.146.189:33909)
0812 14:47:33.376525 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.376527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.376646 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:47:33.376694 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.376695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.376705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.376772 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.377119 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.377879 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.378824 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.379119 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.379209 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.380341 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.380342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.380342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.380348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.382161 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382592 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.382593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.382593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.382639 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382869 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.382870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.382870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.382989 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.383027 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.383211 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.383287 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":18}
I20260812 14:47:33.383422 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.376193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49824)
0812 14:47:33.376346 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.376357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.376437 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.376626 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.376627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.376874 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.376973 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.376974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.377752 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.377752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.378629 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.378631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.379070 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.379070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.379112 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.379626 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.379630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.380103 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.382075 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.382077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.382314 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.382520 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.382521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.382593 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.382870 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.382872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.382915 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.383000 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.383108 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260812 14:47:33.383646 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.376301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53262)
0812 14:47:33.376569 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.376579 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.376640 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.376778 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.376778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.376965 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.377022 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.377023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.377964 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.377965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.378744 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.378746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.379229 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.379230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.379294 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.379809 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.379814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.380261 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.382246 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382248 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.382248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.382491 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.382712 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.382713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.382713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.382790 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.383065 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.383065 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.383102 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.383474 (+ 372us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.383555 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.190@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
I20260812 14:47:33.393862 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.386105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53264 (local address 127.13.146.189:33909)
0812 14:47:33.386144 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.386146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.386294 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:47:33.386378 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.386379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.386386 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.386429 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.386930 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.387342 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.388488 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.388669 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.388726 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.391097 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.391098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.391098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.391104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.392453 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.392955 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.392956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.392956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.393008 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.393261 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.393262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.393267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.393366 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.393441 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.393649 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.393730 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:33.393837 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.385983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53264)
0812 14:47:33.386173 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.386224 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.386303 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.386443 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.386446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.386789 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.386847 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.386849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.387420 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.387421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.388400 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.388402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.388748 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.388750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.388833 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.389330 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.389336 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.391001 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.392537 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.392544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.392546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.392875 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.393073 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.393074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.393075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.393153 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.393457 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.393465 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.393528 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.393623 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.393745 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260812 14:47:33.394635 25410 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.146.189:33909: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9dcf175f04654c318187c2467e7c9556"
member_type: NON_VOTER
last_known_addr {
host: "127.13.146.189"
port: 33909
}
attrs {
promote: true
}
}
dest_uuid: "503ebdeceae3488bbd9bff48396e5e07"
cas_config_opid_index: -1
I20260812 14:47:33.394954 25410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } }
I20260812 14:47:33.395460 26578 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9dcf175f04654c318187c2467e7c9556 (127.13.146.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } }
I20260812 14:47:33.395812 26451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:33.396720 26578 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.146.189:33909
I20260812 14:47:33.396752 26451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 14:47:33.397064 26367 master.cc:537] Master@127.13.146.189:33909 shutting down...
I20260812 14:47:33.397045 26575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } }
I20260812 14:47:33.397140 26575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:33.397563 26578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:47:33.398458 25375 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 14:47:33.398998 25375 consensus_peers.cc:487] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 -> Peer 9dcf175f04654c318187c2467e7c9556 (127.13.146.189:33909): Couldn't send request to peer 9dcf175f04654c318187c2467e7c9556. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:47:33.399890 26367 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:33.399994 26367 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:33.400065 26367 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: stopping tablet replica
I20260812 14:47:33.413044 26367 master.cc:552] Master@127.13.146.189:33909 shutdown complete.
I20260812 14:47:33.413105 26367 master_runner.cc:305] Clearing existing system tablet
I20260812 14:47:33.413277 26367 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.413335 26367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.414011 26367 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:33.414824 26587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.414959 26367 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.415028 26367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:33.415110 26367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "9dcf175f04654c318187c2467e7c9556"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "412bb65d796d9fb09c178611022bda57"
server_key_iv: "c4c45188becb7e81c6ef3ef1c9d3608c"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.415838 26367 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:33.417188 26367 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:47:33.417332 26367 log.cc:1196] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 14:47:33.417551 26367 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Deleting consensus metadata
I20260812 14:47:33.417687 26367 master_runner.cc:315] Copying system tablet from 127.13.146.190:34779
I20260812 14:47:33.418339 26367 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: tablet copy: Beginning tablet copy session from remote peer at address 127.13.146.190:34779
I20260812 14:47:33.425166 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.418728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49826)
0812 14:47:33.418846 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.418858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.418936 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.419082 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.419083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.419264 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.419321 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.419322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.420284 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.420285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.420819 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.420821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.421339 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.421341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.421418 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.421965 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.421969 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.422300 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.424172 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.424173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.424173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.424342 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.424552 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.424553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.424553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.424607 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.424871 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.424872 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.424923 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.424985 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.425095 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":77,"threads_started":1}
I20260812 14:47:33.425256 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.418805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.13.146.190:34779)
0812 14:47:33.418857 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.418859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.418934 (+ 75us) server_negotiation.cc:407] Connection header received
0812 14:47:33.418984 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.418985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.418996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.419070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.419394 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.420192 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.420901 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.421240 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.421314 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.422391 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.422393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.422393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.422402 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.424081 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.424418 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.424419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.424419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.424485 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.424669 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.424670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.424671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.424803 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.424875 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.425036 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.425126 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:47:33.425580 25449 tablet_copy_service.cc:140] P 503ebdeceae3488bbd9bff48396e5e07: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9dcf175f04654c318187c2467e7c9556 ({username='oryx', principal='oryx/127.13.146.189@KRBTEST.COM'} at 127.0.0.1:49826)
I20260812 14:47:33.425720 25449 tablet_copy_service.cc:161] P 503ebdeceae3488bbd9bff48396e5e07: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9dcf175f04654c318187c2467e7c9556 at {username='oryx', principal='oryx/127.13.146.189@KRBTEST.COM'} at 127.0.0.1:49826: session id = 9dcf175f04654c318187c2467e7c9556-00000000000000000000000000000000
I20260812 14:47:33.426717 25449 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:47:33.427439 26367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:47:33.428804 26367 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: tablet copy: Starting download of 0 data blocks...
I20260812 14:47:33.428972 26367 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: tablet copy: Starting download of 1 WAL segments...
I20260812 14:47:33.429979 26367 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:47:33.430914 25449 tablet_copy_service.cc:342] P 503ebdeceae3488bbd9bff48396e5e07: Request end of tablet copy session 9dcf175f04654c318187c2467e7c9556-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.146.189@KRBTEST.COM'} at 127.0.0.1:49826
I20260812 14:47:33.430974 25449 tablet_copy_service.cc:433] P 503ebdeceae3488bbd9bff48396e5e07: ending tablet copy session 9dcf175f04654c318187c2467e7c9556-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9dcf175f04654c318187c2467e7c9556
I20260812 14:47:33.433574 26367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.434705 26605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.434768 26610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.434952 26607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.434986 26367 server_base.cc:1034] running on GCE node
I20260812 14:47:33.435149 26367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.435309 26367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.436434 26367 hybrid_clock.cc:648] HybridClock initialized: now 1786546053436422 us; error 28 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.437933 26367 init.cc:377] Logged in from keytab as oryx/127.13.146.189@KRBTEST.COM (short username oryx)
I20260812 14:47:33.438208 26367 webserver.cc:460] Webserver started at http://127.13.146.189:34723/ using document root <none> and password file <none>
I20260812 14:47:33.438331 26367 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.438367 26367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.439073 26367 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:33.439657 26626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.439814 26367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:33.439870 26367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "9dcf175f04654c318187c2467e7c9556"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "412bb65d796d9fb09c178611022bda57"
server_key_iv: "c4c45188becb7e81c6ef3ef1c9d3608c"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.439929 26367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.454854 26367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.456481 26367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.459767 26367 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.189:33909
I20260812 14:47:33.460032 26737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.189:33909 every 8 connection(s)
I20260812 14:47:33.460222 26367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
I20260812 14:47:33.461433 26738 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:47:33.462002 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Bootstrap starting.
I20260812 14:47:33.465040 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:47:33.465305 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: Bootstrap complete.
I20260812 14:47:33.465539 26738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } }
I20260812 14:47:33.465632 26738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9dcf175f04654c318187c2467e7c9556, State: Initialized, Role: LEARNER
I20260812 14:47:33.465675 26738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } }
I20260812 14:47:33.465914 26745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } }
I20260812 14:47:33.466094 26745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:47:33.465925 26738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:33.468124 26768 catalog_manager.cc:1259] Loaded cluster ID: 297848a9d8cb472eaffcaf4f4d9d2bf1
I20260812 14:47:33.468219 26768 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: loading cluster ID for follower catalog manager: success
I20260812 14:47:33.469084 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.460834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.13.146.189:33909)
0812 14:47:33.460972 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.460973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.460984 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:33.461011 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.461012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.461023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.461124 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.461869 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.462871 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.463837 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.464175 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.464247 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.465826 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.465827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.465827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.465834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.467731 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.468145 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.468146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.468147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.468221 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.468447 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.468448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.468448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.468585 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.468694 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.468900 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.468984 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260812 14:47:33.469635 26768 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: acquiring CA information for follower catalog manager: success
I20260812 14:47:33.470016 26768 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for oryx/127.13.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.188.keytab.
Entry for principal oryx/127.13.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.188.keytab.
I20260812 14:47:33.500533 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:39691
--webserver_interface=127.13.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.188.keytab
--principal=oryx/127.13.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:33.596863 26775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:33.597100 26775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:33.597124 26775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:33.598336 26775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:33.598392 26775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:33.598407 26775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:33.598423 26775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:33.598435 26775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:33.599980 26775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.188.keytab
--principal=oryx/127.13.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:39691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:33.600198 26775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:33.600391 26775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.605516 26786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.605546 26787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.605543 26775 server_base.cc:1034] running on GCE node
W20260812 14:47:33.605526 26794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.606032 26775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.606353 26775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.607546 26775 hybrid_clock.cc:648] HybridClock initialized: now 1786546053607531 us; error 44 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.610298 26775 init.cc:377] Logged in from keytab as oryx/127.13.146.188@KRBTEST.COM (short username oryx)
I20260812 14:47:33.610852 26775 webserver.cc:460] Webserver started at http://127.13.146.188:46345/ using document root <none> and password file <none>
I20260812 14:47:33.611112 26775 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.611183 26775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.611334 26775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:33.612461 26775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/instance:
uuid: "74e982bbcee84593a14b08e33eda24fe"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "ad01138ef822b4f8e7da2c0168d877ab"
server_key_iv: "de9aa75cceeb62462546d4689c4dfb1e"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.612847 26775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal/instance:
uuid: "74e982bbcee84593a14b08e33eda24fe"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "ad01138ef822b4f8e7da2c0168d877ab"
server_key_iv: "de9aa75cceeb62462546d4689c4dfb1e"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.614295 26775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:33.615064 26804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.615242 26775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:33.615353 26775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "74e982bbcee84593a14b08e33eda24fe"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "ad01138ef822b4f8e7da2c0168d877ab"
server_key_iv: "de9aa75cceeb62462546d4689c4dfb1e"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.615427 26775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.640139 26775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.642491 26775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:33.642669 26775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.647540 26775 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.188:39691
I20260812 14:47:33.647594 26906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.188:39691 every 8 connection(s)
I20260812 14:47:33.648032 26775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
I20260812 14:47:33.649080 26909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:33.650722 26909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.188@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:33.657646 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 26775
I20260812 14:47:33.660071 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.651539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.13.146.190:34779)
0812 14:47:33.651603 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.651605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.651717 (+ 112us) server_negotiation.cc:407] Connection header received
0812 14:47:33.652183 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.652184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.652194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.652249 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.652875 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.653494 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.654427 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.654614 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.654668 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.656861 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.656863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.656863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.656870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.658597 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.659021 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.659022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.659023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.659136 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.659410 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.659411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.659411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.659532 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.659607 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.659890 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.659969 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:47:33.660256 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.651411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49832)
0812 14:47:33.651584 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.651627 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.652087 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.652310 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.652313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.652561 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.652755 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.652758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.653581 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.653581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.654348 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.654350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.654704 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.654705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.654801 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.655282 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.655291 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.656771 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.658696 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.658703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.658705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.658937 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.659216 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.659217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.659218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.659332 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.659599 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.659606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.659747 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.659853 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.660014 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260812 14:47:33.661433 26909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.188@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
I20260812 14:47:33.667367 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.658317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.13.146.188:39691)
0812 14:47:33.658509 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.658513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.658530 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:47:33.658548 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.658550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.658560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.658657 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.659238 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.659916 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.660821 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.661309 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.661381 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.664300 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.664307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.664308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.664314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.665981 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.666379 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.666382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.666383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.666451 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.666662 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.666663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.666663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.666875 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.667019 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.667138 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.667214 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260812 14:47:33.669062 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.661726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53276)
0812 14:47:33.661783 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.661794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.661834 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.662133 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.662134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.662251 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.662294 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.662295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.663309 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.663310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.664054 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.664057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.664574 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.664575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.664632 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.664971 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.664976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.666488 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.668302 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.668303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.668303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.668457 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.668612 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.668612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.668612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.668672 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.668855 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.668856 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.668889 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.668938 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.669006 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:47:33.669092 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.661921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.13.146.189:33909)
0812 14:47:33.661994 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.661997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.662006 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:33.662034 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.662035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.662050 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.662117 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.662379 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.663207 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.664171 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.664486 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.664579 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.666574 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.666575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.666575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.666582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.668225 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.668521 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.668521 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.668522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.668573 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.668707 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.668707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.668708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.668801 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.668846 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.668955 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.669018 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:33.669555 26909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.188@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
I20260812 14:47:33.675575 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.669824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37508)
0812 14:47:33.669868 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.669876 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.669936 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.670084 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.670085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.670225 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.670249 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.670250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.670853 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.670854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.671314 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.671316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.671696 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.671697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.671757 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.672071 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.672075 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.673162 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.674602 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.674603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.674603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.674764 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.674969 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.674970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.674970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.675024 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.675269 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.675270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.675306 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.675374 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.675467 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:47:33.675633 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.669870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37508 (local address 127.13.146.188:39691)
0812 14:47:33.669921 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.669923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.669932 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:33.669992 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.669993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.670023 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.670071 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.670321 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.670773 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.671395 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.671603 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.671695 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.673279 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.673280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.673280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.673287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.674505 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.674848 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.674849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.674849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.674904 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.675096 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.675096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.675097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.675201 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.675276 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.675398 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.675516 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":35}
I20260812 14:47:33.677129 26909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Bootstrap starting.
I20260812 14:47:33.678020 26909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:33.678388 26909 log.cc:824] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:33.679152 26909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: No bootstrap required, opened a new log
I20260812 14:47:33.680662 26909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } }
I20260812 14:47:33.680788 26909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:33.680811 26909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e982bbcee84593a14b08e33eda24fe, State: Initialized, Role: FOLLOWER
I20260812 14:47:33.680902 26909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } }
I20260812 14:47:33.681159 26909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:33.681142 26914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } } }
I20260812 14:47:33.681281 26914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:47:33.683070 26937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:33.683149 26937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:33.685638 26775 master_runner.cc:428] Detected that this master 74e982bbcee84593a14b08e33eda24fe is joining an existing cluster
I20260812 14:47:33.685691 26775 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.146.188:39691
I20260812 14:47:33.694252 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.686717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.13.146.189:33909)
0812 14:47:33.687142 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.687144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.687286 (+ 142us) server_negotiation.cc:407] Connection header received
0812 14:47:33.687358 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.687359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.687371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.687428 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.687972 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688710 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.689929 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690221 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690258 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.691331 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.691333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.691333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.691341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.692936 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.693321 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.693322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.693322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.693391 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.693648 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.693649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.693649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.693828 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.693918 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.694062 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.694147 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
I20260812 14:47:33.694269 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.686674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53286)
0812 14:47:33.687209 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.687218 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.687295 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.687480 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.687481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.687797 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.687876 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.687878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688807 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.688807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.689829 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.689831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690337 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.690395 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.690895 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.690899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.691236 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.693021 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.693022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.693022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.693204 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.693462 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.693463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.693463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.693580 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.693909 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.693910 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.693958 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.694011 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.694091 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":72,"thread_start_us":84,"threads_started":1}
I20260812 14:47:33.713671 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.686932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.13.146.188:39691)
0812 14:47:33.686967 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.686968 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.687448 (+ 480us) server_negotiation.cc:407] Connection header received
0812 14:47:33.687525 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.687525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.687534 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.687559 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.687960 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688676 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.689923 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690202 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690243 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.710534 (+ 20291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.710536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.710536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.710543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.712214 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712637 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.712638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.712638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.712715 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712928 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.712928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.712929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.713151 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.713243 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.713407 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.713527 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260812 14:47:33.713927 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.687188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.13.146.190:34779)
0812 14:47:33.687227 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.687229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.687236 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:47:33.687261 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.687262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.687273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.687331 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.687891 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688669 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.689921 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690227 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690271 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.710931 (+ 20660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.710933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.710934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.710944 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.712566 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712977 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.712979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.712979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.713042 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.713327 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.713328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.713328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.713480 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.713548 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.713739 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.713825 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:33.713706 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.686996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37514)
0812 14:47:33.687399 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.687407 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.687467 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.687610 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.687611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.687813 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.687883 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.687885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688766 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.688767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.689829 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.689831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690335 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.690396 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.709977 (+ 19581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.709984 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.710407 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.712313 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.712315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.712545 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.712790 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.712791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.712851 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.713246 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.713247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.713312 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.713384 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.713585 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":174,"thread_start_us":161,"threads_started":1}
I20260812 14:47:33.714304 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.686645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49842)
0812 14:47:33.686822 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.686833 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.686883 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.687382 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.687384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.687700 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.687787 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.687789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.688772 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.688772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.689816 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.689820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.690352 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.690353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.690423 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.710372 (+ 19949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.710378 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.710799 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.712655 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.712656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.712656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.712908 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.713121 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.713122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.713122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.713181 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.713578 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.713579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.713624 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.714077 (+ 453us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.714196 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":39,"thread_start_us":513,"threads_started":1}
I20260812 14:47:33.718716 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 25547
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546052, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.190@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
I20260812 14:47:33.731029 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.716601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37518)
0812 14:47:33.716732 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.716741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.716788 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.716916 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.716917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.717051 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.717093 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.717094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.718004 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.718005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.718814 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.718817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.725435 (+ 6618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.725437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.725531 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.726095 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.726101 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.727790 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.729809 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.729810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.729811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.730065 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.730312 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.730313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.730314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.730387 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.730690 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.730691 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.730743 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.730812 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.730923 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 14:47:33.731384 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.716682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.13.146.188:39691)
0812 14:47:33.716725 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.716726 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.716776 (+ 50us) server_negotiation.cc:407] Connection header received
0812 14:47:33.716848 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.716849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.716859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.716914 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.717161 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.717919 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.724866 (+ 6947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.725329 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.725426 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.727895 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.727897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.727897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.727906 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.729719 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.730156 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.730157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.730158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.730225 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.730454 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.730455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.730456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.730609 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.730669 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.730855 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.730945 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:33.732477 25410 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.146.188:39691: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "74e982bbcee84593a14b08e33eda24fe"
member_type: NON_VOTER
last_known_addr {
host: "127.13.146.188"
port: 39691
}
attrs {
promote: true
}
}
dest_uuid: "503ebdeceae3488bbd9bff48396e5e07"
cas_config_opid_index: 5
I20260812 14:47:33.732758 25410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } }
I20260812 14:47:33.733085 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:44681
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:34779
--webserver_port=36851
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:47:33.733131 26575 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 74e982bbcee84593a14b08e33eda24fe (127.13.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.734215 26575 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.146.188:39691
I20260812 14:47:33.734305 26575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.734380 26575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:33.734869 26775 master.cc:537] Master@127.13.146.188:39691 shutting down...
I20260812 14:47:33.734982 26854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:33.736064 26854 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 14:47:33.736357 26578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:47:33.737133 25370 consensus_peers.cc:487] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 -> Peer 74e982bbcee84593a14b08e33eda24fe (127.13.146.188:39691): Couldn't send request to peer 74e982bbcee84593a14b08e33eda24fe. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:47:33.738893 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.733253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53300)
0812 14:47:33.735232 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.735243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.735311 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.735523 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.735524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.735525 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:33.735565 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.735566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.736511 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.736512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.737738 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.737741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.738625 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.738626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.738701 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.738740 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.738786 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.738824 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1962}
I20260812 14:47:33.739187 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.734535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.13.146.189:33909)
0812 14:47:33.735187 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.735190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.735286 (+ 96us) server_negotiation.cc:407] Connection header received
0812 14:47:33.735385 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.735387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.735409 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.735450 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:33.735639 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.736409 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.737844 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.738448 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.738498 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.738564 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.739035 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.739097 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623}
I20260812 14:47:33.739442 26676 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 LEARNER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 14:47:33.739645 26578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 14:47:33.739816 26775 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:33.739917 26775 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:33.739966 26775 tablet_replica.cc:330] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: stopping tablet replica
I20260812 14:47:33.740278 26676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 74e982bbcee84593a14b08e33eda24fe (127.13.146.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.741448 26745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 503ebdeceae3488bbd9bff48396e5e07. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.741536 26745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:47:33.741569 26745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: NON_VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: true } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.741607 26745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:47:33.742141 26578 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: attempting to promote NON_VOTER 9dcf175f04654c318187c2467e7c9556 to VOTER
I20260812 14:47:33.742337 26578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } }
I20260812 14:47:33.742811 26676 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 LEARNER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 14:47:33.743000 26575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 14:47:33.743593 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.743488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37526)
0812 14:47:33.743525 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.743564 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 14:47:33.743865 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.743799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37530)
0812 14:47:33.743826 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.743846 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 14:47:33.743934 25370 consensus_peers.cc:487] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 -> Peer 74e982bbcee84593a14b08e33eda24fe (127.13.146.188:39691): Couldn't send request to peer 74e982bbcee84593a14b08e33eda24fe. Status: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:47:33.744189 26775 master.cc:552] Master@127.13.146.188:39691 shutdown complete.
I20260812 14:47:33.744282 26775 master_runner.cc:305] Clearing existing system tablet
I20260812 14:47:33.744336 26575 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9dcf175f04654c318187c2467e7c9556 (127.13.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.744648 26775 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.744746 26775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.745285 26575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.745373 26575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:33.745427 26575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.745477 26575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:33.745766 26676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9dcf175f04654c318187c2467e7c9556 (127.13.146.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.746903 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.747004 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:47:33.752018 26775 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260812 14:47:33.752606 26968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.752723 26775 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.752770 26775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:33.752791 26775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "74e982bbcee84593a14b08e33eda24fe"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "ad01138ef822b4f8e7da2c0168d877ab"
server_key_iv: "de9aa75cceeb62462546d4689c4dfb1e"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.754710 26775 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:33.756273 26775 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:47:33.756345 26775 log.cc:1196] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 14:47:33.756541 26775 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Deleting consensus metadata
I20260812 14:47:33.756661 26775 master_runner.cc:315] Copying system tablet from 127.13.146.190:34779
I20260812 14:47:33.757311 26775 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: tablet copy: Beginning tablet copy session from remote peer at address 127.13.146.190:34779
I20260812 14:47:33.764788 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.757604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:49846)
0812 14:47:33.757733 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.757744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.757803 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.757960 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.757961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.758175 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.758224 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.758226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.759150 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.759151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.760008 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.760011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.760556 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.760557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.760625 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.761172 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.761177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.761715 (+ 538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.763575 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.763576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.763576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.763852 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.764092 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.764093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.764093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.764175 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.764461 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.764462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.764512 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.764584 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.764693 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260812 14:47:33.765139 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.757672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.13.146.190:34779)
0812 14:47:33.757718 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.757720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.757789 (+ 69us) server_negotiation.cc:407] Connection header received
0812 14:47:33.757865 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.757866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.757876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.757952 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.758302 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.759062 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.760098 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.760416 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.760492 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.761808 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.761810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.761810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.761818 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.763484 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.763953 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.763954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.763955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.764028 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.764249 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.764249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.764250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.764388 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.764457 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.764627 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.764718 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:33.765336 25449 tablet_copy_service.cc:140] P 503ebdeceae3488bbd9bff48396e5e07: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 74e982bbcee84593a14b08e33eda24fe ({username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:49846)
I20260812 14:47:33.765394 25449 tablet_copy_service.cc:161] P 503ebdeceae3488bbd9bff48396e5e07: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 74e982bbcee84593a14b08e33eda24fe at {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:49846: session id = 74e982bbcee84593a14b08e33eda24fe-00000000000000000000000000000000
I20260812 14:47:33.765955 25449 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:47:33.766607 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:47:33.767925 26775 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: tablet copy: Starting download of 0 data blocks...
I20260812 14:47:33.768164 26775 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: tablet copy: Starting download of 1 WAL segments...
I20260812 14:47:33.769070 26775 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:47:33.770339 25449 tablet_copy_service.cc:342] P 503ebdeceae3488bbd9bff48396e5e07: Request end of tablet copy session 74e982bbcee84593a14b08e33eda24fe-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:49846
I20260812 14:47:33.770391 25449 tablet_copy_service.cc:433] P 503ebdeceae3488bbd9bff48396e5e07: ending tablet copy session 74e982bbcee84593a14b08e33eda24fe-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 74e982bbcee84593a14b08e33eda24fe
I20260812 14:47:33.773761 26775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.775012 26980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.775192 26986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.775632 26982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.775894 26775 server_base.cc:1034] running on GCE node
I20260812 14:47:33.776121 26775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.776260 26775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.777377 26775 hybrid_clock.cc:648] HybridClock initialized: now 1786546053777359 us; error 36 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.779039 26775 init.cc:377] Logged in from keytab as oryx/127.13.146.188@KRBTEST.COM (short username oryx)
I20260812 14:47:33.779313 26775 webserver.cc:460] Webserver started at http://127.13.146.188:34875/ using document root <none> and password file <none>
I20260812 14:47:33.779445 26775 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.779480 26775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.780277 26775 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:33.780995 26999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.781185 26775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:33.781262 26775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "74e982bbcee84593a14b08e33eda24fe"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "ad01138ef822b4f8e7da2c0168d877ab"
server_key_iv: "de9aa75cceeb62462546d4689c4dfb1e"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.781329 26775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.794699 26775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.796576 26775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.799844 26775 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.188:39691
I20260812 14:47:33.800216 26775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
I20260812 14:47:33.801159 27118 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:47:33.801775 27118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Bootstrap starting.
I20260812 14:47:33.804966 27118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:47:33.805261 27118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Bootstrap complete.
I20260812 14:47:33.805270 27113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.188:39691 every 8 connection(s)
I20260812 14:47:33.805478 27118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } }
I20260812 14:47:33.805608 27118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 74e982bbcee84593a14b08e33eda24fe, State: Initialized, Role: LEARNER
I20260812 14:47:33.805661 27118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } }
I20260812 14:47:33.805954 27118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:33.805967 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:33.806223 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: LEARNER
I20260812 14:47:33.807550 27146 catalog_manager.cc:1259] Loaded cluster ID: 297848a9d8cb472eaffcaf4f4d9d2bf1
I20260812 14:47:33.807641 27146 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: loading cluster ID for follower catalog manager: success
I20260812 14:47:33.808948 27146 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: acquiring CA information for follower catalog manager: success
I20260812 14:47:33.809285 27146 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 14:47:33.838260 26957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:33.838475 26957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:33.838500 26957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:33.839749 26957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:33.839818 26957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:33.839862 26957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:33.841706 26957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:44681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=36851
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:33.841976 26957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:33.842226 26957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:33.848686 27160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:33.848688 27162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.848918 26957 server_base.cc:1034] running on GCE node
W20260812 14:47:33.848695 27165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:33.849308 26957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:33.849581 26957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:33.850756 26957 hybrid_clock.cc:648] HybridClock initialized: now 1786546053850738 us; error 31 us; skew 500 ppm
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:33.853905 26957 init.cc:377] Logged in from keytab as oryx/127.13.146.129@KRBTEST.COM (short username oryx)
I20260812 14:47:33.854499 26957 webserver.cc:460] Webserver started at http://127.13.146.129:36851/ using document root <none> and password file <none>
I20260812 14:47:33.854746 26957 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:33.854835 26957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:33.856277 26957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:33.856947 27179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.857177 26957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:33.857318 26957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "cac29c3e72f146a1b79655e2a9fa52e3"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "360b1a5a68037c54357c19fa1b4037c7"
server_key_iv: "ef8ac1f1b2bddda9c0c654b6a86450b5"
server_key_version: "encryptionkey@0"
I20260812 14:47:33.857674 26957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:33.876263 26957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:33.878409 26957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:33.878523 26957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:33.878736 26957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:33.879086 26957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:33.879127 26957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.879150 26957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:33.879199 26957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:33.886353 26957 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44681
I20260812 14:47:33.886399 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44681 every 8 connection(s)
I20260812 14:47:33.886770 26957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:33.891850 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 26957
I20260812 14:47:33.892032 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 25822
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
Aug 12 14:47:33 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546053, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
I20260812 14:47:33.900868 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:40507 (local address 127.13.146.190:34779)
0812 14:47:33.887926 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.887928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.888000 (+ 72us) server_negotiation.cc:407] Connection header received
0812 14:47:33.888533 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.888534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.888548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.888598 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.889451 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890200 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891333 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891668 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891730 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.896930 (+ 5200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.896932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.896932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.896941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.898989 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899645 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.899646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.899647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.899714 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.900039 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.900040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.900040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.900202 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.900245 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.900676 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.900766 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:47:33.900869 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:36461 (local address 127.13.146.188:39691)
0812 14:47:33.887826 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.887828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.888069 (+ 241us) server_negotiation.cc:407] Connection header received
0812 14:47:33.888533 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.888534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.888548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.888598 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.889438 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890090 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891467 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891717 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891777 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.896930 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.896932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.896932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.896941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.899185 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899654 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.899655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.899655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.899714 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.900039 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.900040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.900040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.900186 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.900238 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.900690 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.900775 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":59,"threads_started":1}
I20260812 14:47:33.901460 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:40507)
0812 14:47:33.887902 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.887915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.888427 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.889091 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.889092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.889322 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.889381 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.889383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890280 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.890280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891221 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.891224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891821 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891993 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.892817 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.892822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.896805 (+ 3983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.899093 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899101 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.899104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.899550 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.899794 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.899795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.899955 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.900295 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.900303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.900492 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.900977 (+ 485us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.901200 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":59,"thread_start_us":86,"threads_started":1}
I20260812 14:47:33.901834 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:53997 (local address 127.13.146.189:33909)
0812 14:47:33.887763 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:33.887766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:33.887881 (+ 115us) server_negotiation.cc:407] Connection header received
0812 14:47:33.888512 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:33.888513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:33.888526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:33.888584 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:33.889415 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890193 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891444 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891752 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891817 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.897137 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:33.897139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:33.897139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:33.897149 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:33.900169 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.900695 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.900697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.900697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.900772 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:33.901125 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:33.901126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:33.901127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:33.901341 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:33.901397 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:33.901645 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 14:47:33.901740 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260812 14:47:33.901922 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.13.146.129:53997)
0812 14:47:33.887740 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.887786 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.888401 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.888659 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.888667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.888973 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.889314 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.889319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890502 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.890503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891358 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.891361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891870 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891993 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.892732 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.892741 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.896993 (+ 4252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.900281 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.900282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.900283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.900586 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.900865 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.900866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.900866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.901019 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.901457 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.901458 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.901513 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.901605 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.901739 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":9,"thread_start_us":106,"threads_started":1}
I20260812 14:47:33.902388 27414 heartbeater.cc:347] Connected to a master server at 127.13.146.189:33909
I20260812 14:47:33.902479 27413 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:33.902523 27414 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:33.902648 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:33.887550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.13.146.129:36461)
0812 14:47:33.887996 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:33.888005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:33.888427 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:33.888699 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:33.888700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:33.888994 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:33.889314 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.889322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.890195 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.890196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891358 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:33.891361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:33.891881 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:33.891882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:33.891994 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:33.892964 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:33.892971 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:33.896805 (+ 3834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:33.899271 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.899272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.899550 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.899794 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:33.899795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:33.899795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:33.899955 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:33.900295 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:33.900303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:33.900488 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 14:47:33.901342 (+ 854us) client_negotiation.cc:240] Negotiation successful
0812 14:47:33.901434 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":419,"thread_start_us":74,"threads_started":1}
I20260812 14:47:33.902541 27413 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:33.902812 27414 heartbeater.cc:511] Master 127.13.146.189:33909 requested a full tablet report, sending...
I20260812 14:47:33.902850 27413 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:33.903293 25410 ts_manager.cc:194] Re-registered known tserver with Master: cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129:44681)
I20260812 14:47:33.903469 26652 ts_manager.cc:194] Registered new tserver with Master: cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129:44681)
I20260812 14:47:33.903740 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:40507
I20260812 14:47:33.922842 27412 heartbeater.cc:347] Connected to a master server at 127.13.146.188:39691
I20260812 14:47:33.922946 27412 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:33.923071 27412 heartbeater.cc:511] Master 127.13.146.188:39691 requested a full tablet report, sending...
I20260812 14:47:33.923514 27037 ts_manager.cc:194] Registered new tserver with Master: cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129:44681)
I20260812 14:47:33.926924 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:34839
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:34779
--webserver_port=40905
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:34.012585 27424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:34.012837 27424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:34.012880 27424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:34.014093 27424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:34.014161 27424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:34.014209 27424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:34.015743 27424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:34839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=40905
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:34.016016 27424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:34.016266 27424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:47:34.021611 27424 server_base.cc:1034] running on GCE node
W20260812 14:47:34.021569 27437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:34.021562 27439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:34.021562 27436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:34.022002 27424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:34.022279 27424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:34.023444 27424 hybrid_clock.cc:648] HybridClock initialized: now 1786546054023430 us; error 37 us; skew 500 ppm
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:34.026196 27424 init.cc:377] Logged in from keytab as oryx/127.13.146.130@KRBTEST.COM (short username oryx)
I20260812 14:47:34.026643 27424 webserver.cc:460] Webserver started at http://127.13.146.130:40905/ using document root <none> and password file <none>
I20260812 14:47:34.026870 27424 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:34.026945 27424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:34.028335 27424 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:34.029035 27452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.029309 27424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:34.029417 27424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "a2b0bb258c2d4227a97db8d327158d6b"
format_stamp: "Formatted at 2026-08-12 14:47:32 on dist-test-slave-8hhm"
server_key: "62ffba8fa0e3ade77c16dd5e9d7c1f38"
server_key_iv: "b30d98fffa7b656317bc4358b7e4b031"
server_key_version: "encryptionkey@0"
I20260812 14:47:34.029803 27424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:34.053371 27424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:34.055434 27424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:34.055621 27424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:34.055895 27424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:34.056219 27424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:34.056274 27424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.056315 27424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:34.056344 27424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.062981 27424 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34839
I20260812 14:47:34.063025 27684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34839 every 8 connection(s)
I20260812 14:47:34.063385 27424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:34.072552 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 27424
I20260812 14:47:34.072710 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 26101
I20260812 14:47:34.082577 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:32815 (local address 127.13.146.189:33909)
0812 14:47:34.064704 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.064706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.064714 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:34.065812 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.065813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.065827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.065861 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.066791 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067358 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.068858 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.069140 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069220 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.071726 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.071728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.071728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.071746 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.080703 (+ 8957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.081384 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.081385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.081386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.081455 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.081826 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.081827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.081828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.081974 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.082014 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.082354 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.082442 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:47:34.083338 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.13.146.130:32815)
0812 14:47:34.064467 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.064479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.065701 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.065955 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.065956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.066422 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.066654 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.066660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067467 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.067468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.068773 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.068774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.069238 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.069302 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.069978 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.069983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.071613 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.080841 (+ 9228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.080857 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.080861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.081274 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.081582 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.081582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.081583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.081756 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.082053 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.082058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.082228 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.082959 (+ 731us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.083158 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":241,"thread_start_us":90,"threads_started":1}
I20260812 14:47:34.099272 27693 heartbeater.cc:347] Connected to a master server at 127.13.146.189:33909
I20260812 14:47:34.099396 27693 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.099639 27693 heartbeater.cc:511] Master 127.13.146.189:33909 requested a full tablet report, sending...
I20260812 14:47:34.100180 26652 ts_manager.cc:194] Registered new tserver with Master: a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839)
I20260812 14:47:34.101432 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.13.146.130:34549)
0812 14:47:34.064466 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.064479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.065701 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.065973 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.065974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.066397 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.066654 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.066660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067669 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.067670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.068767 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.068770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.069328 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.069393 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.070059 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.070064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.072122 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.100006 (+ 27884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.100007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.100008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.100278 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.100560 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.100560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.100561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.100632 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.100989 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.100990 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.101039 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.101133 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.101291 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":99,"thread_start_us":96,"threads_started":1}
I20260812 14:47:34.101491 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:34549 (local address 127.13.146.188:39691)
0812 14:47:34.064675 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.064678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.064692 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:34.065812 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.065813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.065827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.065861 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.066774 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067593 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.068880 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.069182 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069270 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.072225 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.072227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.072227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.072237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.081582 (+ 9345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.100395 (+ 18813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.100397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.100398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.100481 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.100708 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.100709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.100709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.100886 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.100939 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.101198 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.101342 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 14:47:34.102025 27688 heartbeater.cc:347] Connected to a master server at 127.13.146.188:39691
I20260812 14:47:34.102087 27688 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.102167 27688 heartbeater.cc:511] Master 127.13.146.188:39691 requested a full tablet report, sending...
I20260812 14:47:34.102452 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:41263 (local address 127.13.146.190:34779)
0812 14:47:34.064726 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.064728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.064739 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:34.065797 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.065798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.065813 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.065875 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.066780 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067550 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.068673 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.068952 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069019 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.097431 (+ 28412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.097434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.097435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.097448 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.100794 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.101186 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.101187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.101188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.101288 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.101743 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.101745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.101745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.101919 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.101995 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.102215 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.102313 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260812 14:47:34.102473 27037 ts_manager.cc:194] Registered new tserver with Master: a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839)
I20260812 14:47:34.102819 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.064205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:41263)
0812 14:47:34.064399 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.064444 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.065682 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.065952 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.065955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.066419 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.066653 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.066658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.067664 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.067665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.068576 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.068579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.069036 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.069037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.069196 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.069911 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.069922 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.072724 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.100870 (+ 28146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.100871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.100871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.101094 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.101368 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.101369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.101369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.101425 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.102017 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.102018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.102066 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.102174 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.102291 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":81,"thread_start_us":109,"threads_started":1}
I20260812 14:47:34.103220 27690 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:34.103279 27690 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.103349 27690 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:34.103627 25410 ts_manager.cc:194] Re-registered known tserver with Master: a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839)
I20260812 14:47:34.104020 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:41263
I20260812 14:47:34.104635 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:36391
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:34779
--webserver_port=38987
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:47:34.166093 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.162713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37536)
0812 14:47:34.162783 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.162798 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.162856 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.163116 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.163118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.163120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.163178 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.163180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.164180 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.164181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.165117 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.165119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.165899 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.165900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.165948 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.165951 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.165989 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.166028 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 14:47:34.166110 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.162824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.13.146.188:39691)
0812 14:47:34.162903 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.162905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.162916 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:34.162950 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.162951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.162976 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.163068 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.163283 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.164068 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.165186 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.165820 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.165905 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.165984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.166004 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.166037 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:47:34.166474 27055 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 14:47:34.166642 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 503ebdeceae3488bbd9bff48396e5e07. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:34.166817 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: LEARNER
W20260812 14:47:34.190752 27703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:34.190955 27703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:34.190979 27703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:34.192093 27703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:34.192132 27703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:34.192158 27703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:34.193799 27703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:36391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=38987
--tserver_master_addrs=127.13.146.190:34779,127.13.146.189:33909,127.13.146.188:39691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:34.194072 27703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:34.194365 27703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:34.199602 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:34.199590 27717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:34.199877 27703 server_base.cc:1034] running on GCE node
W20260812 14:47:34.199610 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:34.200155 27703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:34.200402 27703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:34.201546 27703 hybrid_clock.cc:648] HybridClock initialized: now 1786546054201536 us; error 30 us; skew 500 ppm
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:34.204152 27703 init.cc:377] Logged in from keytab as oryx/127.13.146.131@KRBTEST.COM (short username oryx)
I20260812 14:47:34.204631 27703 webserver.cc:460] Webserver started at http://127.13.146.131:38987/ using document root <none> and password file <none>
I20260812 14:47:34.204887 27703 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:34.204981 27703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:34.206539 27703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:34.207226 27741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.207422 27703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:34.207521 27703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "c1372e01bfde4a21823e90d1412d6539"
format_stamp: "Formatted at 2026-08-12 14:47:33 on dist-test-slave-8hhm"
server_key: "82ec26a93b41d92c3ace84d3f4572d4d"
server_key_iv: "cc60a716bc12b0f7c765d8e45a8318aa"
server_key_version: "encryptionkey@0"
I20260812 14:47:34.207877 27703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:34.214763 27703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:34.216856 27703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:34.217002 27703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:34.217308 27703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:34.217643 27703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:34.217684 27703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.217730 27703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:34.217757 27703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:34.223930 27703 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36391
I20260812 14:47:34.224020 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36391 every 8 connection(s)
I20260812 14:47:34.224467 27703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:34.230253 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 27703
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.189@KRBTEST.COM
Aug 12 14:47:34 dist-test-slave-8hhm krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546054, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.188@KRBTEST.COM
I20260812 14:47:34.239513 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:58157 (local address 127.13.146.189:33909)
0812 14:47:34.225715 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.225717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.225960 (+ 243us) server_negotiation.cc:407] Connection header received
0812 14:47:34.226489 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.226490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.226502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.226561 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.227778 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228626 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.230370 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.230661 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.230738 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.233833 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.233835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.233835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.233843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.235420 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.236004 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.236005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.236005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.236090 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.236835 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.236837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.236837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.236984 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.237054 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.239296 (+ 2242us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.239391 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:47:34.241089 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:50121 (local address 127.13.146.190:34779)
0812 14:47:34.225706 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.225709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.226773 (+ 1064us) server_negotiation.cc:407] Connection header received
0812 14:47:34.226845 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.226846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.226856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.226926 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.227753 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228260 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.229364 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.229574 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.229653 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.237454 (+ 7801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.237455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.237456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.237463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.239663 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.240091 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.240093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.240093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.240160 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.240402 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.240403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.240403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.240566 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.240612 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.240854 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.240959 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:47:34.241688 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:50121)
0812 14:47:34.226694 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.226705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.226771 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.226958 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.226960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.227447 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.227651 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.227656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228375 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.228376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.229283 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.229286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.229660 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.229661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.229795 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.230643 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.230654 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.235516 (+ 4862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.239759 (+ 4243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.239760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.239760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.239994 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.240249 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.240250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.240251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.240321 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.240683 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.240684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.240733 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.240820 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.241071 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260812 14:47:34.242457 28003 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34779
I20260812 14:47:34.242595 28003 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.242841 28003 heartbeater.cc:511] Master 127.13.146.190:34779 requested a full tablet report, sending...
I20260812 14:47:34.242892 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:56253 (local address 127.13.146.188:39691)
0812 14:47:34.225654 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.225656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.226964 (+ 1308us) server_negotiation.cc:407] Connection header received
0812 14:47:34.227079 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.227080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.227093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.227151 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.227777 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228619 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.229914 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.230229 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.230303 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.238904 (+ 8601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.238907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.238907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.238916 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.240978 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.241577 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.241578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.241579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.241634 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.242205 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.242206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.242207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.242356 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.242419 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.242713 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.242798 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:47:34.243229 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.13.146.131:56253)
0812 14:47:34.226895 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.226905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.226994 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.227156 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.227157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.227476 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.227652 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.227660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228737 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.228738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.229825 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.229827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.230303 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.230304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.230385 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.233886 (+ 3501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.233892 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.236723 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.241061 (+ 4338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.241062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.241063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.241301 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.241720 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.241721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.241721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.241791 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.242449 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.242451 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.242493 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.242760 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.242868 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"mutex_wait_us":123,"thread_start_us":127,"threads_started":1}
I20260812 14:47:34.243279 25410 ts_manager.cc:194] Re-registered known tserver with Master: c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:34.243790 25410 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:50121
I20260812 14:47:34.244020 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.231443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.13.146.190:34779)
0812 14:47:34.232476 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.232478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.232486 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:34.232514 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.232515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.232530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.232621 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.233469 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.234253 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.236292 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.236588 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.236644 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.238411 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.238412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.238412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.238419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.242341 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.242786 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.242787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.242788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.242854 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.243176 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.243178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.243178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.243322 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.243426 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.243811 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.243908 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":78,"threads_started":1}
I20260812 14:47:34.244145 28002 heartbeater.cc:347] Connected to a master server at 127.13.146.188:39691
I20260812 14:47:34.244189 28002 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.244262 28002 heartbeater.cc:511] Master 127.13.146.188:39691 requested a full tablet report, sending...
I20260812 14:47:34.244593 27037 ts_manager.cc:194] Registered new tserver with Master: c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:34.244755 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.231569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.13.146.188:39691)
0812 14:47:34.232696 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.232698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.232707 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:34.232732 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.232733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.232747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.234388 (+ 1641us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.234435 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.235285 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.236347 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.236647 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.238396 (+ 1749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.238452 (+ 56us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.238453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.238453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.238458 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.243045 (+ 4587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.243459 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.243459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.243460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.243521 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.243743 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.243743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.243744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.243881 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.244006 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.244503 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.244587 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":70,"threads_started":1}
I20260812 14:47:34.245872 28009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.225341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.13.146.131:58157)
0812 14:47:34.225821 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.225862 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.226391 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.226708 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.226713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.227468 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:34.227652 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.227660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.228737 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.228738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.229821 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.229824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.230799 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.230801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.230878 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.231578 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:34.231584 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:34.233723 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:34.235601 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.235609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.235611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.235904 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.236171 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:34.236172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:34.236172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:34.236362 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:34.237092 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:34.237099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:34.237298 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.244934 (+ 7636us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.245407 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":68,"thread_start_us":205,"threads_started":1}
I20260812 14:47:34.246218 28005 heartbeater.cc:347] Connected to a master server at 127.13.146.189:33909
I20260812 14:47:34.246279 28005 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:34.246340 28005 heartbeater.cc:511] Master 127.13.146.189:33909 requested a full tablet report, sending...
I20260812 14:47:34.246727 26652 ts_manager.cc:194] Registered new tserver with Master: c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:34.246877 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.231875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.13.146.189:33909)
0812 14:47:34.232869 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.232871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.232880 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:34.232901 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.232902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.232913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.233851 (+ 938us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.233897 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.236045 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.237331 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.237709 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.238042 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.239356 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.239358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.239358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.239365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.242061 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.245570 (+ 3509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.245572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.245572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.245645 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.245892 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.245893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.245894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.246072 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.246610 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.246658 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.246756 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":57,"threads_started":1}
I20260812 14:47:34.246340 25410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:34.248222 25410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:34.248998 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: NON_VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: true } } }
I20260812 14:47:34.249104 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:34.250325 26964 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: attempting to promote NON_VOTER 74e982bbcee84593a14b08e33eda24fe to VOTER
I20260812 14:47:34.250453 26964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } }
I20260812 14:47:34.250952 27055 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 LEARNER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 14:47:34.251194 28039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 14:47:34.251452 26671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Refusing update from remote peer 503ebdeceae3488bbd9bff48396e5e07: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 14:47:34.251849 28039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 14:47:34.252149 26964 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 74e982bbcee84593a14b08e33eda24fe (127.13.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.252363 27055 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 74e982bbcee84593a14b08e33eda24fe (127.13.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.252641 26671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 74e982bbcee84593a14b08e33eda24fe (127.13.146.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.253150 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.253262 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:34.253321 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.253377 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:34.253419 26964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.253476 26964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:34.253741 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.253825 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:47:34.254333 28058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "503ebdeceae3488bbd9bff48396e5e07" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:34.254518 28058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:47:34.258111 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.254437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34839 (local address 127.0.0.1:40214)
0812 14:47:34.254577 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.254589 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.254636 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.255150 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.255150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.255152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.255182 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.255183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.256267 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.256268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.256985 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.256987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.257878 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.257879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.257950 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.257954 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.257997 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.258036 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":2}
I20260812 14:47:34.258124 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.254774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.13.146.130:34839)
0812 14:47:34.254975 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.254980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.254991 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:34.255032 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.255034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.255050 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.255360 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.255484 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.256172 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.257081 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.257816 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.257913 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.257980 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.258010 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.258045 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260812 14:47:34.258881 27552 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b (DEFAULT_TABLE table=test-table [id=4d53d13e21e94347ae7707bf6337f48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:34.259292 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:34.259473 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.254557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57272 (local address 127.13.146.131:36391)
0812 14:47:34.254907 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.254914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.254926 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:34.254989 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.254991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.255008 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.255097 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.255308 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.256567 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.258104 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.258947 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.259023 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.259092 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.259332 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.259380 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":101,"threads_started":1}
I20260812 14:47:34.259780 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.254436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36391 (local address 127.0.0.1:57272)
0812 14:47:34.254855 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.254865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.254914 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.255090 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.255091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.255094 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.255153 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.255155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.256689 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.256690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.257982 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.257984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.259166 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.259167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.259239 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.259242 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.259625 (+ 383us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.259682 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":281,"thread_start_us":93,"threads_started":1}
I20260812 14:47:34.260080 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.254488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44681 (local address 127.0.0.1:48166)
0812 14:47:34.254697 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.254710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.254770 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.255508 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.255509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.255511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.255583 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.255585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.257391 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.257392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.258603 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.258606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.259841 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.259842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.259907 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.259911 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.259971 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.260012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260812 14:47:34.260152 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.255005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.13.146.129:44681)
0812 14:47:34.255316 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.255320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.255330 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:34.255372 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.255374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.255392 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.255670 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.255735 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.257285 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.258868 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.259726 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.259833 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.259902 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.260002 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.260054 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
I20260812 14:47:34.260583 27852 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b (DEFAULT_TABLE table=test-table [id=4d53d13e21e94347ae7707bf6337f48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:34.260802 27270 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b (DEFAULT_TABLE table=test-table [id=4d53d13e21e94347ae7707bf6337f48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:34.261004 27852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:34.261164 27270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:34.263142 28070 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Bootstrap starting.
I20260812 14:47:34.263818 28070 tablet_bootstrap.cc:654] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:34.264145 28070 log.cc:824] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:34.264997 28070 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: No bootstrap required, opened a new log
I20260812 14:47:34.265082 28070 ts_tablet_manager.cc:1397] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:34.265413 28075 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Bootstrap starting.
I20260812 14:47:34.266152 28077 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Bootstrap starting.
I20260812 14:47:34.267151 28075 tablet_bootstrap.cc:654] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:34.267364 28077 tablet_bootstrap.cc:654] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:34.267511 28070 raft_consensus.cc:348] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.267735 28070 raft_consensus.cc:374] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:34.267756 28077 log.cc:824] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:34.267779 28070 raft_consensus.cc:729] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b0bb258c2d4227a97db8d327158d6b, State: Initialized, Role: FOLLOWER
I20260812 14:47:34.267928 28070 consensus_queue.cc:260] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.268332 28070 ts_tablet_manager.cc:1428] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:34.268447 28075 log.cc:824] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:34.268445 27690 heartbeater.cc:503] Master 127.13.146.190:34779 was elected leader, sending a full tablet report...
I20260812 14:47:34.269968 28077 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: No bootstrap required, opened a new log
I20260812 14:47:34.270043 28077 ts_tablet_manager.cc:1397] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260812 14:47:34.270452 28075 tablet_bootstrap.cc:492] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: No bootstrap required, opened a new log
I20260812 14:47:34.270558 28075 ts_tablet_manager.cc:1397] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 14:47:34.271914 28077 raft_consensus.cc:348] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.272043 28077 raft_consensus.cc:374] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:34.272066 28077 raft_consensus.cc:729] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac29c3e72f146a1b79655e2a9fa52e3, State: Initialized, Role: FOLLOWER
I20260812 14:47:34.272192 28077 consensus_queue.cc:260] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.272475 28077 ts_tablet_manager.cc:1428] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:47:34.272487 27413 heartbeater.cc:503] Master 127.13.146.190:34779 was elected leader, sending a full tablet report...
I20260812 14:47:34.273516 28075 raft_consensus.cc:348] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.273756 28075 raft_consensus.cc:374] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:34.273808 28075 raft_consensus.cc:729] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1372e01bfde4a21823e90d1412d6539, State: Initialized, Role: FOLLOWER
I20260812 14:47:34.273932 28075 consensus_queue.cc:260] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.274305 28003 heartbeater.cc:503] Master 127.13.146.190:34779 was elected leader, sending a full tablet report...
I20260812 14:47:34.274235 28075 ts_tablet_manager.cc:1428] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260812 14:47:34.314522 27694 tablet.cc:2369] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:34.373804 28085 raft_consensus.cc:482] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:34.373945 28085 raft_consensus.cc:504] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.374425 28085 leader_election.cc:288] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839), c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:34.378779 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.374718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:36985 (local address 127.13.146.131:36391)
0812 14:47:34.374779 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.374782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.374794 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:34.374857 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.374857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.374867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.374925 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.375174 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.375945 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.377760 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.378323 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.378399 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.378407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.378657 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.378707 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:47:34.378779 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.374736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:40587 (local address 127.13.146.130:34839)
0812 14:47:34.374791 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.374794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.374802 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:34.374833 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.374834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.374844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.374922 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:34.375174 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.375936 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.377745 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.378321 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.378399 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.378407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.378657 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.378707 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:47:34.378769 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.374648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36391 (local address 127.13.146.129:36985)
0812 14:47:34.374754 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.374763 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.374795 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.374949 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.374950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.374952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.375073 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.375075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.376092 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.376093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.377636 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.377640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.378423 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.378425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.378532 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.378537 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.378611 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.378660 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":173}
I20260812 14:47:34.378769 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.374636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34839 (local address 127.13.146.129:40587)
0812 14:47:34.374690 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:34.374701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:34.374749 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:34.374949 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:34.374950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:34.374952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:34.375073 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.375075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.376067 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.376068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.377636 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:34.377640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.378423 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.378425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:34.378535 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.378537 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:34.378611 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:47:34.378660 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":138}
I20260812 14:47:34.379254 27594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc46a8a96e4c1cb3d82295c26e7f6b" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b0bb258c2d4227a97db8d327158d6b" is_pre_election: true
I20260812 14:47:34.379333 27899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc46a8a96e4c1cb3d82295c26e7f6b" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1372e01bfde4a21823e90d1412d6539" is_pre_election: true
I20260812 14:47:34.379470 27594 raft_consensus.cc:2454] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 0.
I20260812 14:47:34.379470 27899 raft_consensus.cc:2454] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 0.
I20260812 14:47:34.379750 27180 leader_election.cc:302] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1372e01bfde4a21823e90d1412d6539, cac29c3e72f146a1b79655e2a9fa52e3; no voters:
I20260812 14:47:34.379936 28085 raft_consensus.cc:2781] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:34.380043 28085 raft_consensus.cc:482] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:34.380080 28085 raft_consensus.cc:3037] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:34.380838 28085 raft_consensus.cc:504] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.381016 28085 leader_election.cc:288] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 election: Requested vote from peers a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839), c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:34.381258 27594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc46a8a96e4c1cb3d82295c26e7f6b" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b0bb258c2d4227a97db8d327158d6b"
I20260812 14:47:34.381204 27899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc46a8a96e4c1cb3d82295c26e7f6b" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1372e01bfde4a21823e90d1412d6539"
I20260812 14:47:34.381349 27899 raft_consensus.cc:3037] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:34.381348 27594 raft_consensus.cc:3037] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:34.382309 27594 raft_consensus.cc:2454] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 1.
I20260812 14:47:34.382309 27899 raft_consensus.cc:2454] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 1.
I20260812 14:47:34.382494 27185 leader_election.cc:302] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b0bb258c2d4227a97db8d327158d6b, cac29c3e72f146a1b79655e2a9fa52e3; no voters:
I20260812 14:47:34.382601 28085 raft_consensus.cc:2781] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:34.382718 28085 raft_consensus.cc:686] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 LEADER]: Becoming Leader. State: Replica: cac29c3e72f146a1b79655e2a9fa52e3, State: Running, Role: LEADER
I20260812 14:47:34.382826 28085 consensus_queue.cc:237] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:34.383708 25410 catalog_manager.cc:5495] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129). New cstate: current_term: 1 leader_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } health_report { overall_health: UNKNOWN } } }
W20260812 14:47:34.388513 27416 tablet.cc:2369] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:34.404774 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.402283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.13.146.129:44681)
0812 14:47:34.402345 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.402347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.402357 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:34.402412 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.402413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.402433 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.402515 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:34.402633 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.403116 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.403913 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.404114 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.404195 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.404396 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:34.404510 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:34.404599 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.404644 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.404666 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:47:34.407050 27594 raft_consensus.cc:1261] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Refusing update from remote peer cac29c3e72f146a1b79655e2a9fa52e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:34.407099 27899 raft_consensus.cc:1261] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Refusing update from remote peer cac29c3e72f146a1b79655e2a9fa52e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:34.407485 28089 consensus_queue.cc:1035] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:34.407709 28085 consensus_queue.cc:1035] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:34.414005 28095 mvcc.cc:205] Tried to move back new op lower bound from 7317692638848000000 to 7317692638752788480. Current Snapshot: MvccSnapshot[applied={T|T < 7317692638848000000 or (T in {7317692638848000000})}]
I20260812 14:47:34.414381 25410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:34.414491 25410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:34.415947 25410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 503ebdeceae3488bbd9bff48396e5e07: Sending DeleteTablet for 3 replicas of tablet 89bc46a8a96e4c1cb3d82295c26e7f6b
I20260812 14:47:34.416379 27552 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:34 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:40214
I20260812 14:47:34.416422 27852 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:34 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57272
I20260812 14:47:34.416684 28116 tablet_replica.cc:330] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: stopping tablet replica
I20260812 14:47:34.416813 28116 raft_consensus.cc:2229] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:34.416927 28116 raft_consensus.cc:2258] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:34.417034 27270 tablet_service.cc:1514] Processing DeleteTablet for tablet 89bc46a8a96e4c1cb3d82295c26e7f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:34 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:48166
I20260812 14:47:34.417512 28116 ts_tablet_manager.cc:1905] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:34.417711 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 25334
I20260812 14:47:34.429811 28116 ts_tablet_manager.cc:1918] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:34.429919 28116 log.cc:1196] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/89bc46a8a96e4c1cb3d82295c26e7f6b
I20260812 14:47:34.430262 28116 ts_tablet_manager.cc:1939] T 89bc46a8a96e4c1cb3d82295c26e7f6b P c1372e01bfde4a21823e90d1412d6539: Deleting consensus metadata
I20260812 14:47:34.430550 28118 tablet_replica.cc:330] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: stopping tablet replica
I20260812 14:47:34.430704 28118 raft_consensus.cc:2229] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:34.430842 28118 raft_consensus.cc:2258] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:34.430889 28115 tablet_replica.cc:330] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: stopping tablet replica
I20260812 14:47:34.431013 28115 raft_consensus.cc:2229] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:34.431139 28115 raft_consensus.cc:2258] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:34.432403 28118 ts_tablet_manager.cc:1905] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:34.432422 28115 ts_tablet_manager.cc:1905] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:34.434233 28118 ts_tablet_manager.cc:1918] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:34.434237 28115 ts_tablet_manager.cc:1918] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:34.434324 28118 log.cc:1196] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/89bc46a8a96e4c1cb3d82295c26e7f6b
I20260812 14:47:34.434324 28115 log.cc:1196] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/89bc46a8a96e4c1cb3d82295c26e7f6b
I20260812 14:47:34.434646 28115 ts_tablet_manager.cc:1939] T 89bc46a8a96e4c1cb3d82295c26e7f6b P a2b0bb258c2d4227a97db8d327158d6b: Deleting consensus metadata
I20260812 14:47:34.434646 28118 ts_tablet_manager.cc:1939] T 89bc46a8a96e4c1cb3d82295c26e7f6b P cac29c3e72f146a1b79655e2a9fa52e3: Deleting consensus metadata
I20260812 14:47:34.440125 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.417316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.13.146.188:39691)
0812 14:47:34.417365 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.417368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.430086 (+ 12718us) server_negotiation.cc:407] Connection header received
0812 14:47:34.430262 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.430264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.430285 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.430323 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.431518 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.432457 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.433928 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.434339 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.434390 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.436532 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.436534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.436534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.436545 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.438550 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.439048 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.439050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.439051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.439186 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.439452 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.439453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.439454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.439620 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.439737 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.439893 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.440000 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 14:47:34.440801 27185 connection.cc:535] server connection from 127.0.0.1:48166 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:47:34.440889 27454 connection.cc:535] server connection from 127.0.0.1:40214 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:47:34.440915 27747 connection.cc:535] server connection from 127.0.0.1:57272 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:47:34.451500 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.442936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.13.146.188:39691)
0812 14:47:34.442989 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.442991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.443289 (+ 298us) server_negotiation.cc:407] Connection header received
0812 14:47:34.443395 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.443396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.443408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.443470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.444477 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.445017 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.445894 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.446120 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.446200 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.447919 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.447921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.447921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.447929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.450000 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.450561 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.450563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.450563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.450631 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.450881 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.450882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.450883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.451033 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.451153 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.451278 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.451360 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:47:34.451886 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:34.443126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.13.146.189:33909)
0812 14:47:34.443171 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:34.443173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:34.443231 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:47:34.443327 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:34.443328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:34.443346 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:34.443408 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:34.444261 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.445140 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.446353 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:34.446636 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:34.446738 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:34.448205 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:34.448207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:34.448207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:34.448215 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:34.450027 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.450585 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.450586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.450587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.450657 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:34.451189 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:34.451191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:34.451191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:34.451342 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:34.451445 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:34.451661 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 14:47:34.451783 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:47:35.413829 28134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.413330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:41409)
0812 14:47:35.413705 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.413791 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":75,"threads_started":1}
I20260812 14:47:35.413827 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.413535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:34659)
0812 14:47:35.413690 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.413769 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":45,"threads_started":1}
W20260812 14:47:35.414075 27413 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:34779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
W20260812 14:47:35.414075 28003 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:34779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.414448 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.414212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:38603)
0812 14:47:35.414329 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.414413 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
W20260812 14:47:35.414685 27690 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:34779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.916297 28139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 503ebdeceae3488bbd9bff48396e5e07)
I20260812 14:47:35.916440 28139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } }
I20260812 14:47:35.916714 28139 leader_election.cc:288] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779), 9dcf175f04654c318187c2467e7c9556 (127.13.146.189:33909)
I20260812 14:47:35.917251 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.916845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50348)
0812 14:47:35.916983 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.917038 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260812 14:47:35.917281 28153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 503ebdeceae3488bbd9bff48396e5e07)
I20260812 14:47:35.917424 28153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } }
W20260812 14:47:35.917537 27000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.917770 28153 leader_election.cc:288] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779), 74e982bbcee84593a14b08e33eda24fe (127.13.146.188:39691)
I20260812 14:47:35.918038 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.917839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50362)
0812 14:47:35.917978 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.918008 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260812 14:47:35.918151 27000 leader_election.cc:334] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779): Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.918350 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.917852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50366)
0812 14:47:35.918020 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.918311 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":63,"threads_started":1}
W20260812 14:47:35.918462 26627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.918877 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.918768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50374)
0812 14:47:35.918818 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.918852 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:47:35.919015 26627 leader_election.cc:334] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779): Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.920470 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.916902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:33909 (local address 127.0.0.1:53338)
0812 14:47:35.917030 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.917042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:35.917156 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:35.917346 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:35.917346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:35.917350 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:35.917411 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.917414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.918584 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.918586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.919638 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.919641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.920205 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.920205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.920280 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.920284 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:35.920362 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:47:35.920407 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260812 14:47:35.920468 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.917001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.13.146.189:33909)
0812 14:47:35.917171 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:35.917174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:35.917185 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:35.917237 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:35.917238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:35.917250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:35.917329 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:35.917489 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.918487 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.919730 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.920117 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.920206 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.920215 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:35.920370 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 14:47:35.920408 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260812 14:47:35.920683 26671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e982bbcee84593a14b08e33eda24fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9dcf175f04654c318187c2467e7c9556" is_pre_election: true
I20260812 14:47:35.920830 26671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e982bbcee84593a14b08e33eda24fe in term 1.
I20260812 14:47:35.920941 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.917874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39691 (local address 127.0.0.1:37582)
0812 14:47:35.918142 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.918153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:35.918220 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:35.918377 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:35.918378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:35.918380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:35.918426 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.918428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.919304 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.919305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.920066 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.920068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.920692 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.920692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.920770 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.920773 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:35.920817 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:47:35.920861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260812 14:47:35.920975 27003 leader_election.cc:302] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e982bbcee84593a14b08e33eda24fe, 9dcf175f04654c318187c2467e7c9556; no voters: 503ebdeceae3488bbd9bff48396e5e07
I20260812 14:47:35.921098 28139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 14:47:35.921128 28139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader 503ebdeceae3488bbd9bff48396e5e07)
I20260812 14:47:35.921164 28139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:47:35.921250 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.917980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37582 (local address 127.13.146.188:39691)
0812 14:47:35.918110 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:35.918112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:35.918223 (+ 111us) server_negotiation.cc:407] Connection header received
0812 14:47:35.918272 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:35.918273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:35.918283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:35.918355 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:35.918515 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.919219 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.920160 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.920576 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.920650 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.920658 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:35.921061 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 14:47:35.921111 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260812 14:47:35.921442 27055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9dcf175f04654c318187c2467e7c9556" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "74e982bbcee84593a14b08e33eda24fe" is_pre_election: true
I20260812 14:47:35.922022 28139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } }
I20260812 14:47:35.922183 28139 leader_election.cc:288] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 election: Requested vote from peers 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779), 9dcf175f04654c318187c2467e7c9556 (127.13.146.189:33909)
I20260812 14:47:35.922189 27055 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9dcf175f04654c318187c2467e7c9556 in current term 2: Already voted for candidate 74e982bbcee84593a14b08e33eda24fe in this term.
I20260812 14:47:35.922358 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.922268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50386)
0812 14:47:35.922302 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.922334 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:47:35.922328 26671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74e982bbcee84593a14b08e33eda24fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9dcf175f04654c318187c2467e7c9556"
I20260812 14:47:35.922402 26628 leader_election.cc:302] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dcf175f04654c318187c2467e7c9556; no voters: 503ebdeceae3488bbd9bff48396e5e07, 74e982bbcee84593a14b08e33eda24fe
I20260812 14:47:35.922392 26671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:47:35.922637 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.922556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50392)
0812 14:47:35.922586 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.922614 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 14:47:35.923244 26671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e982bbcee84593a14b08e33eda24fe in term 2.
I20260812 14:47:35.923389 28153 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 14:47:35.923380 27000 leader_election.cc:334] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779): Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
I20260812 14:47:35.923461 27003 leader_election.cc:302] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e982bbcee84593a14b08e33eda24fe, 9dcf175f04654c318187c2467e7c9556; no voters: 503ebdeceae3488bbd9bff48396e5e07
I20260812 14:47:35.923560 28139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:47:35.923826 28139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [term 2 LEADER]: Becoming Leader. State: Replica: 74e982bbcee84593a14b08e33eda24fe, State: Running, Role: LEADER
I20260812 14:47:35.923897 28139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } }
I20260812 14:47:35.924252 28163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e982bbcee84593a14b08e33eda24fe. Latest consensus state: current_term: 2 leader_uuid: "74e982bbcee84593a14b08e33eda24fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:35.924336 28163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:35.924523 28168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:35.925055 28168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:35.925169 28168 catalog_manager.cc:1259] Loaded cluster ID: 297848a9d8cb472eaffcaf4f4d9d2bf1
I20260812 14:47:35.925203 28168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:35.925395 28168 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:35.925498 28168 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Loaded TSK: 0
I20260812 14:47:35.925639 28168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:35.937295 27037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:35.938876 27037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:35.939992 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.939898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50410)
0812 14:47:35.939933 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.939963 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:47:35.940052 26671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [term 2 FOLLOWER]: Refusing update from remote peer 74e982bbcee84593a14b08e33eda24fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 14:47:35.940292 28139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 14:47:35.940426 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.940319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.0.0.1:50426)
0812 14:47:35.940354 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.940405 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 14:47:35.940598 27000 consensus_peers.cc:487] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe -> Peer 503ebdeceae3488bbd9bff48396e5e07 (127.13.146.190:34779): Couldn't send request to peer 503ebdeceae3488bbd9bff48396e5e07. Status: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:47:35.940961 28153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e982bbcee84593a14b08e33eda24fe. Latest consensus state: current_term: 2 leader_uuid: "74e982bbcee84593a14b08e33eda24fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:35.941066 28153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:47:35.941656 28171 mvcc.cc:205] Tried to move back new op lower bound from 7317692645128314880 to 7317692645064896512. Current Snapshot: MvccSnapshot[applied={T|T < 7317692645128314880}]
I20260812 14:47:35.941742 28169 mvcc.cc:205] Tried to move back new op lower bound from 7317692645128314880 to 7317692645064896512. Current Snapshot: MvccSnapshot[applied={T|T < 7317692645128314880}]
I20260812 14:47:35.941838 28153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "74e982bbcee84593a14b08e33eda24fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:35.941854 28139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "74e982bbcee84593a14b08e33eda24fe" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "503ebdeceae3488bbd9bff48396e5e07" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34779 } } peers { permanent_uuid: "9dcf175f04654c318187c2467e7c9556" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 33909 } attrs { promote: false } } peers { permanent_uuid: "74e982bbcee84593a14b08e33eda24fe" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39691 } attrs { promote: false } } }
I20260812 14:47:35.941929 28153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dcf175f04654c318187c2467e7c9556 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:47:35.941931 28139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:35.947129 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.943763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44681 (local address 127.0.0.1:48184)
0812 14:47:35.943793 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.943804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:35.943850 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:35.944359 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:35.944359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:35.944360 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:35.944390 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.944391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.945062 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.945063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.946170 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.946173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.946904 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.946905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.946964 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.946969 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:35.947025 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:47:35.947065 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 14:47:35.947182 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.943848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.13.146.129:44681)
0812 14:47:35.944106 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:35.944108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:35.944117 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:35.944146 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:35.944147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:35.944157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:35.944224 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:35.944467 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.944956 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.946248 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.946807 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.946883 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.946892 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:35.947043 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:47:35.947087 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":66,"threads_started":1}
I20260812 14:47:35.947455 27270 tablet_service.cc:1467] Processing CreateTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 (DEFAULT_TABLE table=test-table [id=28c25eefb9c249b2a9bcbc9b5940b5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:35.947615 27270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a8ca5d44c401fb2d671224b5d0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:35.947768 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.943908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36391 (local address 127.0.0.1:57274)
0812 14:47:35.944028 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.944038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:35.944090 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:35.944487 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:35.944488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:35.944490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:35.944529 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.944531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.945440 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.945441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.946499 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.946501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.947538 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.947539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.947601 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.947606 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:35.947660 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:47:35.947703 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
I20260812 14:47:35.948007 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.944087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.13.146.131:36391)
0812 14:47:35.944346 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:35.944348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:35.944356 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:35.944387 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:35.944388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:35.944399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:35.944589 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:35.944627 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.945362 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.946568 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.947137 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.947590 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.947600 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:35.947711 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:47:35.947753 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":53,"threads_started":1}
I20260812 14:47:35.948225 27852 tablet_service.cc:1467] Processing CreateTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 (DEFAULT_TABLE table=test-table [id=28c25eefb9c249b2a9bcbc9b5940b5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:35.948292 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.943841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34839 (local address 127.0.0.1:40224)
0812 14:47:35.943916 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.943925 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:35.943964 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:35.944232 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:35.944233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:35.944235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:35.944285 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.944287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.945362 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.945363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.947327 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:35.947348 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.948095 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.948096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:35.948156 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.948159 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:35.948188 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 14:47:35.948228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1018}
I20260812 14:47:35.948378 27852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a8ca5d44c401fb2d671224b5d0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:35.948565 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.943978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.13.146.130:34839)
0812 14:47:35.944107 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:35.944109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:35.944117 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:35.944146 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:35.944147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:35.944158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:35.944214 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:35.944468 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.945266 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.947459 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:35.947981 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:35.948406 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:35.948416 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:35.948438 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 14:47:35.948477 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260812 14:47:35.949095 28189 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: Bootstrap starting.
I20260812 14:47:35.949306 27552 tablet_service.cc:1467] Processing CreateTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 (DEFAULT_TABLE table=test-table [id=28c25eefb9c249b2a9bcbc9b5940b5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:35.949455 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a8ca5d44c401fb2d671224b5d0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:35.949586 28189 tablet_bootstrap.cc:654] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:35.950291 28189 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: No bootstrap required, opened a new log
I20260812 14:47:35.950345 28189 ts_tablet_manager.cc:1397] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:35.950488 28189 raft_consensus.cc:348] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.950557 28189 raft_consensus.cc:374] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:35.950572 28189 raft_consensus.cc:729] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac29c3e72f146a1b79655e2a9fa52e3, State: Initialized, Role: FOLLOWER
I20260812 14:47:35.950644 28189 consensus_queue.cc:260] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.950784 28189 ts_tablet_manager.cc:1428] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:35.951054 28196 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Bootstrap starting.
I20260812 14:47:35.951256 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.951012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:44221)
0812 14:47:35.951173 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.951217 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260812 14:47:35.951455 28196 tablet_bootstrap.cc:654] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:35.951463 28193 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Bootstrap starting.
I20260812 14:47:35.951756 27412 heartbeater.cc:503] Master 127.13.146.188:39691 was elected leader, sending a full tablet report...
I20260812 14:47:35.951829 28193 tablet_bootstrap.cc:654] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:35.952099 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.951564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:49055)
0812 14:47:35.951973 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.952006 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391}
I20260812 14:47:35.952529 28196 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: No bootstrap required, opened a new log
I20260812 14:47:35.952598 28196 ts_tablet_manager.cc:1397] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:35.952696 28193 tablet_bootstrap.cc:492] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: No bootstrap required, opened a new log
I20260812 14:47:35.952754 28193 ts_tablet_manager.cc:1397] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:35.952765 28196 raft_consensus.cc:348] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.952840 28196 raft_consensus.cc:374] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:35.952870 28196 raft_consensus.cc:729] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b0bb258c2d4227a97db8d327158d6b, State: Initialized, Role: FOLLOWER
I20260812 14:47:35.952921 28193 raft_consensus.cc:348] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.952940 28196 consensus_queue.cc:260] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.952996 28193 raft_consensus.cc:374] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:35.953011 28193 raft_consensus.cc:729] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1372e01bfde4a21823e90d1412d6539, State: Initialized, Role: FOLLOWER
I20260812 14:47:35.953125 28196 ts_tablet_manager.cc:1428] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:35.953081 28193 consensus_queue.cc:260] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:35.953508 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.953289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:32855)
0812 14:47:35.953420 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.953457 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260812 14:47:35.953552 28193 ts_tablet_manager.cc:1428] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:35.953680 27688 heartbeater.cc:503] Master 127.13.146.188:39691 was elected leader, sending a full tablet report...
I20260812 14:47:35.953856 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.953741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:54723)
0812 14:47:35.953805 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.953834 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 14:47:35.954348 28002 heartbeater.cc:503] Master 127.13.146.188:39691 was elected leader, sending a full tablet report...
I20260812 14:47:35.954420 28212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.954157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:59035)
0812 14:47:35.954302 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.954341 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
I20260812 14:47:35.954838 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:35.954635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:42509)
0812 14:47:35.954758 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:35.954797 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":84,"threads_started":1}
W20260812 14:47:35.978006 28006 tablet.cc:2369] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:36.082463 28194 raft_consensus.cc:482] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:36.082612 28194 raft_consensus.cc:504] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:36.082854 28194 leader_election.cc:288] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839), c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:36.083120 27594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a8ca5d44c401fb2d671224b5d0005" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b0bb258c2d4227a97db8d327158d6b" is_pre_election: true
I20260812 14:47:36.083159 27899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a8ca5d44c401fb2d671224b5d0005" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1372e01bfde4a21823e90d1412d6539" is_pre_election: true
I20260812 14:47:36.083252 27594 raft_consensus.cc:2454] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 0.
I20260812 14:47:36.083266 27899 raft_consensus.cc:2454] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 0.
I20260812 14:47:36.083441 27180 leader_election.cc:302] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1372e01bfde4a21823e90d1412d6539, cac29c3e72f146a1b79655e2a9fa52e3; no voters:
I20260812 14:47:36.083572 28194 raft_consensus.cc:2781] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:36.083617 28194 raft_consensus.cc:482] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:36.083642 28194 raft_consensus.cc:3037] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:36.084368 28194 raft_consensus.cc:504] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:36.084512 28194 leader_election.cc:288] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 election: Requested vote from peers a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839), c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391)
I20260812 14:47:36.084697 27899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a8ca5d44c401fb2d671224b5d0005" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1372e01bfde4a21823e90d1412d6539"
I20260812 14:47:36.084720 27594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a8ca5d44c401fb2d671224b5d0005" candidate_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b0bb258c2d4227a97db8d327158d6b"
I20260812 14:47:36.084789 27594 raft_consensus.cc:3037] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:36.084789 27899 raft_consensus.cc:3037] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:36.085655 27594 raft_consensus.cc:2454] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 1.
I20260812 14:47:36.085655 27899 raft_consensus.cc:2454] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cac29c3e72f146a1b79655e2a9fa52e3 in term 1.
I20260812 14:47:36.085865 27180 leader_election.cc:302] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a2b0bb258c2d4227a97db8d327158d6b, c1372e01bfde4a21823e90d1412d6539, cac29c3e72f146a1b79655e2a9fa52e3; no voters:
I20260812 14:47:36.085973 28194 raft_consensus.cc:2781] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:36.086158 28194 raft_consensus.cc:686] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 LEADER]: Becoming Leader. State: Replica: cac29c3e72f146a1b79655e2a9fa52e3, State: Running, Role: LEADER
I20260812 14:47:36.086237 28194 consensus_queue.cc:237] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } }
I20260812 14:47:36.086838 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.086707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:46511)
0812 14:47:36.086755 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.086796 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:47:36.086894 27037 catalog_manager.cc:5495] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129). New cstate: current_term: 1 leader_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cac29c3e72f146a1b79655e2a9fa52e3" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:36.129181 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.126830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.13.146.129:44681)
0812 14:47:36.126897 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:36.126899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:36.126908 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:36.126957 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:36.126958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:36.126967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:36.127015 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:36.127222 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:36.127721 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:36.128509 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:36.128701 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:36.128758 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:36.128884 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:36.128911 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:36.128949 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:36.129002 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:47:36.129023 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:47:36.130257 27594 raft_consensus.cc:1261] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Refusing update from remote peer cac29c3e72f146a1b79655e2a9fa52e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:36.130259 27899 raft_consensus.cc:1261] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Refusing update from remote peer cac29c3e72f146a1b79655e2a9fa52e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:36.130534 28218 consensus_queue.cc:1035] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b0bb258c2d4227a97db8d327158d6b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:36.130682 28194 consensus_queue.cc:1035] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1372e01bfde4a21823e90d1412d6539" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:36.131609 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.131494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:53963)
0812 14:47:36.131533 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.131574 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 14:47:36.131708 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.131401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:60475)
0812 14:47:36.131636 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.131673 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260812 14:47:36.132488 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.132383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:53451)
0812 14:47:36.132430 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.132457 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:47:36.133039 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.132937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.130:35301)
0812 14:47:36.132980 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.133008 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 14:47:36.133577 27036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:36.133680 27036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:36.134050 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.133953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.129:38975)
0812 14:47:36.133994 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.134019 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:47:36.134073 28231 mvcc.cc:205] Tried to move back new op lower bound from 7317692645907521536 to 7317692645729775616. Current Snapshot: MvccSnapshot[applied={T|T < 7317692645907521536}]
I20260812 14:47:36.134447 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.134348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34779 (local address 127.13.146.131:58857)
0812 14:47:36.134392 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.134418 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:47:36.134572 28230 mvcc.cc:205] Tried to move back new op lower bound from 7317692645907521536 to 7317692645729775616. Current Snapshot: MvccSnapshot[applied={T|T < 7317692645907521536 or (T in {7317692645907521536})}]
I20260812 14:47:36.135387 27036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74e982bbcee84593a14b08e33eda24fe: Sending DeleteTablet for 3 replicas of tablet d37a8ca5d44c401fb2d671224b5d0005
I20260812 14:47:36.135596 27852 tablet_service.cc:1514] Processing DeleteTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:36 UTC) from {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:57274
I20260812 14:47:36.135628 27270 tablet_service.cc:1514] Processing DeleteTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:36 UTC) from {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:48184
I20260812 14:47:36.135717 27552 tablet_service.cc:1514] Processing DeleteTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:36 UTC) from {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:40224
I20260812 14:47:36.135859 28259 tablet_replica.cc:330] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3: stopping tablet replica
I20260812 14:47:36.135881 27852 tablet_service.cc:1514] Processing DeleteTablet for tablet d37a8ca5d44c401fb2d671224b5d0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:36 UTC) from {username='oryx', principal='oryx/127.13.146.188@KRBTEST.COM'} at 127.0.0.1:57274
I20260812 14:47:36.135948 28258 tablet_replica.cc:330] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: stopping tablet replica
I20260812 14:47:36.135998 28258 raft_consensus.cc:2229] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:36.136049 28258 raft_consensus.cc:2258] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:36.136014 28259 raft_consensus.cc:2229] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:36.136144 28259 raft_consensus.cc:2258] T d37a8ca5d44c401fb2d671224b5d0005 P cac29c3e72f146a1b79655e2a9fa52e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:36.136197 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 26957
I20260812 14:47:36.136309 28260 tablet_replica.cc:330] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: stopping tablet replica
I20260812 14:47:36.136385 28260 raft_consensus.cc:2229] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:36.136425 28260 raft_consensus.cc:2258] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:36.142146 28260 ts_tablet_manager.cc:1905] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:36.143831 28260 ts_tablet_manager.cc:1918] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:36.143891 28260 log.cc:1196] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/d37a8ca5d44c401fb2d671224b5d0005
I20260812 14:47:36.144080 28260 ts_tablet_manager.cc:1939] T d37a8ca5d44c401fb2d671224b5d0005 P a2b0bb258c2d4227a97db8d327158d6b: Deleting consensus metadata
I20260812 14:47:36.144510 27004 catalog_manager.cc:4841] TS a2b0bb258c2d4227a97db8d327158d6b (127.13.146.130:34839): tablet d37a8ca5d44c401fb2d671224b5d0005 (table test-table [id=28c25eefb9c249b2a9bcbc9b5940b5e8]) successfully deleted
I20260812 14:47:36.147149 28258 ts_tablet_manager.cc:1905] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:36.148624 28258 ts_tablet_manager.cc:1918] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:36.148710 28258 log.cc:1196] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/d37a8ca5d44c401fb2d671224b5d0005
I20260812 14:47:36.148917 28258 ts_tablet_manager.cc:1939] T d37a8ca5d44c401fb2d671224b5d0005 P c1372e01bfde4a21823e90d1412d6539: Deleting consensus metadata
I20260812 14:47:36.149328 27000 catalog_manager.cc:4841] TS c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391): tablet d37a8ca5d44c401fb2d671224b5d0005 (table test-table [id=28c25eefb9c249b2a9bcbc9b5940b5e8]) successfully deleted
W20260812 14:47:36.149492 27000 catalog_manager.cc:4808] TS c1372e01bfde4a21823e90d1412d6539 (127.13.146.131:36391): delete failed for tablet d37a8ca5d44c401fb2d671224b5d0005 because the tablet was not found. No further retry: Not found: Tablet not found: d37a8ca5d44c401fb2d671224b5d0005
I20260812 14:47:36.153745 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:36.153611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44681 (local address 127.0.0.1:48216)
0812 14:47:36.153675 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:36.153714 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:47:36.153848 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 27424
W20260812 14:47:36.153942 27003 catalog_manager.cc:4584] TS cac29c3e72f146a1b79655e2a9fa52e3 (127.13.146.129:44681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d37a8ca5d44c401fb2d671224b5d0005: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44681: connect: Connection refused (error 111)
I20260812 14:47:36.164209 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 27703
I20260812 14:47:36.176469 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 26367
I20260812 14:47:36.187183 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 26775
2026-08-12T14:47:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5194 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:36 dist-test-slave-8hhm krb5kdc[28268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:36 dist-test-slave-8hhm krb5kdc[28268](info): set up 2 sockets
Aug 12 14:47:36 dist-test-slave-8hhm krb5kdc[28268](info): commencing operation
krb5kdc: starting...
W20260812 14:47:37.504837 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:37 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab.
Entry for principal oryx/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab.
I20260812 14:47:37.632047 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:43491
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab
--principal=oryx/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:43491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:37.715497 28311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:37.715760 28311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:37.715801 28311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:37.716980 28311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:37.717051 28311 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:47:37.717075 28311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:37.717108 28311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:37.717135 28311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:37.717164 28311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:37.718806 28311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:43491
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.190.keytab
--principal=oryx/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:43491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:37.719085 28311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:37.719316 28311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:37.724308 28322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:37.724308 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:37.724366 28320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:37.724692 28311 server_base.cc:1034] running on GCE node
I20260812 14:47:37.724992 28311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:37.725340 28311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:37.726493 28311 hybrid_clock.cc:648] HybridClock initialized: now 1786546057726475 us; error 32 us; skew 500 ppm
Aug 12 14:47:37 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:37.729154 28311 init.cc:377] Logged in from keytab as oryx/127.13.146.190@KRBTEST.COM (short username oryx)
I20260812 14:47:37.729661 28311 webserver.cc:460] Webserver started at http://127.13.146.190:40559/ using document root <none> and password file <none>
I20260812 14:47:37.729915 28311 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:37.729970 28311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:37.730155 28311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:37.731244 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "d535937cfb98413ba21d4e14a8c4eee9"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "0d80e117f44d578ccbafa7cf76c62b47"
server_key_iv: "05dad2d48c4ecedf6858b144a36c0663"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.731607 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "d535937cfb98413ba21d4e14a8c4eee9"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "0d80e117f44d578ccbafa7cf76c62b47"
server_key_iv: "05dad2d48c4ecedf6858b144a36c0663"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.732990 28311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 14:47:37.733886 28343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:37.734083 28311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:37.734175 28311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "d535937cfb98413ba21d4e14a8c4eee9"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "0d80e117f44d578ccbafa7cf76c62b47"
server_key_iv: "05dad2d48c4ecedf6858b144a36c0663"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.734252 28311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:37.751845 28311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:37.753903 28311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:37.754047 28311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:37.759040 28311 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:43491
I20260812 14:47:37.759083 28460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:43491 every 8 connection(s)
I20260812 14:47:37.759404 28311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:37.760083 28461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:37.762671 28461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Bootstrap starting.
I20260812 14:47:37.763340 28461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:37.763670 28461 log.cc:824] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:37.764379 28461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: No bootstrap required, opened a new log
I20260812 14:47:37.766589 28461 raft_consensus.cc:348] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } }
I20260812 14:47:37.766746 28461 raft_consensus.cc:374] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:37.766772 28461 raft_consensus.cc:729] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d535937cfb98413ba21d4e14a8c4eee9, State: Initialized, Role: FOLLOWER
I20260812 14:47:37.766976 28461 consensus_queue.cc:260] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } }
I20260812 14:47:37.767055 28461 raft_consensus.cc:388] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:37.767098 28461 raft_consensus.cc:482] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:37.767153 28461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:37.767691 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 28311
I20260812 14:47:37.767798 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:37.767927 28461 raft_consensus.cc:504] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } }
I20260812 14:47:37.768064 13898 external_mini_cluster.cc:1428] Setting key 27aacb3dde677da6e1858de55cec016d
I20260812 14:47:37.768062 28461 leader_election.cc:302] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d535937cfb98413ba21d4e14a8c4eee9; no voters:
I20260812 14:47:37.768240 28461 leader_election.cc:288] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:37.768304 28467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:37.768466 28467 raft_consensus.cc:686] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [term 1 LEADER]: Becoming Leader. State: Replica: d535937cfb98413ba21d4e14a8c4eee9, State: Running, Role: LEADER
I20260812 14:47:37.768550 28461 sys_catalog.cc:564] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:37.768572 28467 consensus_queue.cc:237] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } }
I20260812 14:47:37.768952 28467 sys_catalog.cc:455] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d535937cfb98413ba21d4e14a8c4eee9. Latest consensus state: current_term: 1 leader_uuid: "d535937cfb98413ba21d4e14a8c4eee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } } }
I20260812 14:47:37.769048 28467 sys_catalog.cc:458] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:37.768960 28469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d535937cfb98413ba21d4e14a8c4eee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d535937cfb98413ba21d4e14a8c4eee9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 43491 } } }
I20260812 14:47:37.769306 28469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:37.769315 28483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:37.769995 28483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:37.771622 28483 catalog_manager.cc:1347] Generated new cluster ID: 6517306ee66a455fa60edda82e14cb8f
I20260812 14:47:37.771684 28483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:37 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:37.779799 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:37.768759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.13.146.190:43491)
0812 14:47:37.768958 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:37.768968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:37.769714 (+ 746us) server_negotiation.cc:407] Connection header received
0812 14:47:37.769863 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:37.769871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:37.769942 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:37.770779 (+ 837us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:37.771156 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.772280 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.773055 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.773548 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.773665 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:37.776025 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:37.776029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:37.776029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:37.776042 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:37.778291 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.778655 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.778658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.778659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.778756 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.778962 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.778962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.778963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.779208 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:37.779388 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:37.779467 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 14:47:37.779625 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260812 14:47:37.785356 28483 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:37.785892 28483 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:37.796296 28483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Generated new TSK 0
I20260812 14:47:37.796483 28483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
I20260812 14:47:37.832203 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:43491
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:37.934530 28516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:37.934758 28516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:37.934783 28516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:37.935946 28516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:37.935987 28516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:37.936029 28516 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:37.936064 28516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:37.937786 28516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:37.938040 28516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:37.938268 28516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:37.943293 28522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:37.943257 28525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:37.943256 28529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:37.943643 28516 server_base.cc:1034] running on GCE node
I20260812 14:47:37.943843 28516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:37.944087 28516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:37.945271 28516 hybrid_clock.cc:648] HybridClock initialized: now 1786546057945260 us; error 32 us; skew 500 ppm
Aug 12 14:47:37 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:37.947995 28516 init.cc:377] Logged in from keytab as oryx/127.13.146.129@KRBTEST.COM (short username oryx)
I20260812 14:47:37.948463 28516 webserver.cc:460] Webserver started at http://127.13.146.129:45173/ using document root <none> and password file <none>
I20260812 14:47:37.948719 28516 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:37.948783 28516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:37.948954 28516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:37.950062 28516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "5a89456fff9909cbbf0c9e3660f00421"
server_key_iv: "7cc0ef1d302b68e8c595b7573c14cb00"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.950413 28516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "5a89456fff9909cbbf0c9e3660f00421"
server_key_iv: "7cc0ef1d302b68e8c595b7573c14cb00"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.951843 28516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:37.952670 28544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:37.952844 28516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:37.952943 28516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
format_stamp: "Formatted at 2026-08-12 14:47:37 on dist-test-slave-8hhm"
server_key: "5a89456fff9909cbbf0c9e3660f00421"
server_key_iv: "7cc0ef1d302b68e8c595b7573c14cb00"
server_key_version: "encryptionkey@0"
I20260812 14:47:37.953023 28516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:37.969590 28516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:37.971653 28516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:37.971828 28516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:37.972446 28516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:37.972504 28516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:37.972537 28516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:37.972558 28516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:47:37 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:37.979979 28516 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44679
I20260812 14:47:37.980113 28782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44679 every 8 connection(s)
I20260812 14:47:37.980425 28516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:37.982343 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:37.972927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:45255 (local address 127.13.146.190:43491)
0812 14:47:37.972997 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:37.972999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:37.973187 (+ 188us) server_negotiation.cc:407] Connection header received
0812 14:47:37.973748 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:37.973749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:37.973770 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:37.973844 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:37.974590 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.975104 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.976161 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.976397 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.976472 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:37.979308 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:37.979310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:37.979311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:37.979319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:37.981043 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.981499 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.981500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.981501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.981565 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.981812 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.981813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.981813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.981920 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:37.981980 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:37.982213 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 14:47:37.982280 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:47:37.982463 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:37.972895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.129:45255)
0812 14:47:37.973055 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:37.973095 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:37.973650 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:37.973967 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:37.973970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:37.974273 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:37.974461 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:37.974465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.975208 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.975209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:37.976049 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:37.976051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.976481 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.976482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:37.976611 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:37.977113 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:37.977121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:37.979202 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:37.981111 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:37.981116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:37.981117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:37.981433 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:37.981621 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:37.981621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:37.981621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:37.981727 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:37.981970 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:37.981972 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:37.982100 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 14:47:37.982170 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:47:37.982286 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260812 14:47:37.983011 28785 heartbeater.cc:347] Connected to a master server at 127.13.146.190:43491
I20260812 14:47:37.983109 28785 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:37.983299 28785 heartbeater.cc:511] Master 127.13.146.190:43491 requested a full tablet report, sending...
I20260812 14:47:37.983817 28371 ts_manager.cc:194] Registered new tserver with Master: 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679)
I20260812 14:47:37.984520 28371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:45255
I20260812 14:47:37.988560 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 28516
I20260812 14:47:37.988649 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:37.988878 13898 external_mini_cluster.cc:1428] Setting key 70a36f45d5b323e19526b41c4ada2e0b
I20260812 14:47:37.990146 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:37.984038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:39289 (local address 127.13.146.190:43491)
0812 14:47:37.984720 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:37.984723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:37.984731 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:37.984751 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:37.984752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:37.984762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:37.984813 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:37.985192 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.985713 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.986393 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.986591 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.986657 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:37.987532 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:37.987533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:37.987533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:37.987539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:37.989008 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.989414 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.989415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.989415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.989475 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:37.989657 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:37.989657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:37.989658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:37.989785 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:37.989856 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:37.990009 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:47:37.990073 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662}
I20260812 14:47:37.990249 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:37.983992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.129:39289)
0812 14:47:37.984131 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:37.984140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:37.984186 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:37.984821 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:37.984822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:37.985035 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:37.985099 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:37.985101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.985806 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.985806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:37.986323 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:37.986325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:37.986692 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:37.986694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:37.986776 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:37.987148 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:37.987152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:37.987452 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:37.989087 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:37.989088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:37.989089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:37.989342 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:37.989542 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:37.989543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:37.989543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:37.989600 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:37.989859 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:37.989860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:37.989907 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:47:37.989993 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:47:37.990100 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
I20260812 14:47:38.016686 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:43491
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:38.100346 28807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:38.100591 28807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:38.100630 28807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:38.101851 28807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:38.101907 28807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:38.101940 28807 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:38.101979 28807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:38.103472 28807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:38.103737 28807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:38.103966 28807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:38.108896 28823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:38.108886 28815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:38.108906 28813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:38.109274 28807 server_base.cc:1034] running on GCE node
I20260812 14:47:38.109505 28807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:38.109767 28807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:38.110910 28807 hybrid_clock.cc:648] HybridClock initialized: now 1786546058110896 us; error 30 us; skew 500 ppm
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:38.113600 28807 init.cc:377] Logged in from keytab as oryx/127.13.146.130@KRBTEST.COM (short username oryx)
I20260812 14:47:38.114059 28807 webserver.cc:460] Webserver started at http://127.13.146.130:39519/ using document root <none> and password file <none>
I20260812 14:47:38.114274 28807 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:38.114315 28807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:38.114403 28807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:38.115368 28807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "751a62f5e6de40dd810c35500413e5aa"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "34c8c66d7cf11196b7704a3d1a8b0d5d"
server_key_iv: "d443522fe7335746b421886cbb19f902"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.115679 28807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "751a62f5e6de40dd810c35500413e5aa"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "34c8c66d7cf11196b7704a3d1a8b0d5d"
server_key_iv: "d443522fe7335746b421886cbb19f902"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.117039 28807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:47:38.117782 28836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.117965 28807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.118057 28807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "751a62f5e6de40dd810c35500413e5aa"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "34c8c66d7cf11196b7704a3d1a8b0d5d"
server_key_iv: "d443522fe7335746b421886cbb19f902"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.118140 28807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:38.127856 28807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:38.129699 28807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:38.129845 28807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:38.130437 28807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:38.130491 28807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.130534 28807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:38.130568 28807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:38.137771 28807 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:40393
I20260812 14:47:38.137861 29070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:40393 every 8 connection(s)
I20260812 14:47:38.138147 28807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:38.140816 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.130780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:54549 (local address 127.13.146.190:43491)
0812 14:47:38.130842 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.130845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.131069 (+ 224us) server_negotiation.cc:407] Connection header received
0812 14:47:38.131594 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.131596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.131617 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.131679 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.132669 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.133431 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.134591 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.134803 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.134881 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.137741 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.137743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.137747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.137771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.139394 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.139803 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.139805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.139805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.139868 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.140122 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.140123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.140123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.140238 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.140337 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.140633 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.140728 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:47:38.140981 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.130773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.130:54549)
0812 14:47:38.130941 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.130980 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.131488 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.131807 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.131812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.132254 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:38.132558 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.132565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.133536 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.133537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.134484 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.134487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.134908 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.134909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.135025 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.135523 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:38.135531 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:38.137644 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:38.139494 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.139501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.139504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.139728 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.139932 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.139932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.139933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.140044 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.140328 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:38.140336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:38.140473 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.140594 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.140757 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260812 14:47:38.141502 29072 heartbeater.cc:347] Connected to a master server at 127.13.146.190:43491
I20260812 14:47:38.141607 29072 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:38.141779 29072 heartbeater.cc:511] Master 127.13.146.190:43491 requested a full tablet report, sending...
I20260812 14:47:38.142210 28371 ts_manager.cc:194] Registered new tserver with Master: 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393)
I20260812 14:47:38.142306 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 28807
I20260812 14:47:38.142424 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:38.142635 13898 external_mini_cluster.cc:1428] Setting key 1ee2ec4756db3bbc9d5a601730a12777
I20260812 14:47:38.142712 28371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:54549
I20260812 14:47:38.150902 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.143069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:41325 (local address 127.13.146.190:43491)
0812 14:47:38.143110 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.143112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.143436 (+ 324us) server_negotiation.cc:407] Connection header received
0812 14:47:38.145261 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.145262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.145274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.145335 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.145679 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.146154 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.146916 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.147130 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.147224 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.148203 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.148204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.148205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.148212 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.149683 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.150044 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.150046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.150046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.150113 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.150316 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.150317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.150317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.150467 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.150555 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.150727 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.150805 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:38.150910 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.143011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.130:41325)
0812 14:47:38.143384 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.143396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.143449 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.145353 (+ 1904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.145355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.145533 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:38.145588 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.145590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.146261 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.146262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.146825 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.146827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.147227 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.147228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.147297 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.147751 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:38.147758 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:38.148106 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:38.149782 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.149784 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.149784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.149961 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.150180 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.150181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.150181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.150242 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.150547 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:38.150548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:38.150614 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.150700 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.150807 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
I20260812 14:47:38.170342 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:43491
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:38.253113 29091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:38.253329 29091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:38.253355 29091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:38.254515 29091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:38.254559 29091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:38.254583 29091 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:47:38.254644 29091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:38.256170 29091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.146.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:38.256454 29091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:38.256682 29091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:38.261857 29110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:38.261969 29091 server_base.cc:1034] running on GCE node
W20260812 14:47:38.261911 29098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:38.261873 29102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:38.262348 29091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:38.262616 29091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:38.263765 29091 hybrid_clock.cc:648] HybridClock initialized: now 1786546058263751 us; error 41 us; skew 500 ppm
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:38.266469 29091 init.cc:377] Logged in from keytab as oryx/127.13.146.131@KRBTEST.COM (short username oryx)
I20260812 14:47:38.266959 29091 webserver.cc:460] Webserver started at http://127.13.146.131:44257/ using document root <none> and password file <none>
I20260812 14:47:38.267231 29091 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:38.267298 29091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:38.267445 29091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:38.268574 29091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "e981b286bd8442fa9503162e88328893"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "d8c5323cb2ecfbd1864f9ae1b8fe6205"
server_key_iv: "77186e457d1afb1ceb38d17b1366c808"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.269017 29091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "e981b286bd8442fa9503162e88328893"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "d8c5323cb2ecfbd1864f9ae1b8fe6205"
server_key_iv: "77186e457d1afb1ceb38d17b1366c808"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.270475 29091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:47:38.271185 29122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.271358 29091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:38.271463 29091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "e981b286bd8442fa9503162e88328893"
format_stamp: "Formatted at 2026-08-12 14:47:38 on dist-test-slave-8hhm"
server_key: "d8c5323cb2ecfbd1864f9ae1b8fe6205"
server_key_iv: "77186e457d1afb1ceb38d17b1366c808"
server_key_version: "encryptionkey@0"
I20260812 14:47:38.271541 29091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:38.279525 29091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:38.281672 29091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:38.281831 29091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:38.282516 29091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:38.282568 29091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.282604 29091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:38.282622 29091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:38.290464 29091 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:45961
I20260812 14:47:38.290552 29356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:45961 every 8 connection(s)
I20260812 14:47:38.290922 29091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:38.292742 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.283411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:39289 (local address 127.13.146.190:43491)
0812 14:47:38.284233 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.284236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.284247 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:38.284279 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.284280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.284292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.284354 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.285171 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.285804 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.286862 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.287089 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.287175 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.289820 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.289822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.289822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.289830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.291542 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.291944 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.291946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.291946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.292009 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.292247 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.292248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.292248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.292361 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.292419 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.292607 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.292673 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771}
I20260812 14:47:38.292953 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.283085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.131:39289)
0812 14:47:38.283246 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.283297 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.284026 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.284496 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.284501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.284857 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:38.285057 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.285063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.285904 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.285905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.286763 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.286765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.287181 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.287182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.287326 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.287908 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:38.287920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:38.289711 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:38.291622 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.291627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.291628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.291855 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.292104 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.292105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.292105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.292193 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.292410 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:38.292412 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:38.292517 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.292591 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.292716 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260812 14:47:38.293488 29359 heartbeater.cc:347] Connected to a master server at 127.13.146.190:43491
I20260812 14:47:38.293587 29359 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:38.293766 29359 heartbeater.cc:511] Master 127.13.146.190:43491 requested a full tablet report, sending...
I20260812 14:47:38.294171 28376 ts_manager.cc:194] Registered new tserver with Master: e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:38.294711 28376 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:39289
I20260812 14:47:38.295660 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 29091
I20260812 14:47:38.295753 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:38.295934 13898 external_mini_cluster.cc:1428] Setting key f2ef181698c6d1fbac65b0cb92d4482f
I20260812 14:47:38.297104 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:47:38.300705 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.294517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.13.146.131:43675)
0812 14:47:38.294651 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.294673 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.294734 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.294899 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.294900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.295132 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:38.295199 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.295201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.296041 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.296042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.296817 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.296820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.297295 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.297296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.297360 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.297747 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:38.297753 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:38.298163 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:38.299746 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.299747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.299754 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.299918 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.300135 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.300136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.300136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.300194 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.300472 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:38.300472 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:38.300505 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.300557 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.300637 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260812 14:47:38.300809 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.294646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:43675 (local address 127.13.146.190:43491)
0812 14:47:38.294705 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.294707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.294718 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:38.294794 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.294795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.294805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.294854 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.295282 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.295905 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.296886 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.297169 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.297291 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.298250 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.298252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.298252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.298258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.299671 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.299998 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.300000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.300000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.300067 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.300264 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.300265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.300265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.300406 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.300486 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.300594 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.300692 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:47:38.324184 28472 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:38.315696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.13.146.190:43491)
0812 14:47:38.315771 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.315773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.315784 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:38.315824 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.315825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.315845 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.315911 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.316529 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.317011 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.317799 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.317978 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.318034 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.324087 (+ 6053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39064: BlockingRecv error: recv got EOF from 127.0.0.1:39064 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:38.331815 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.324965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.13.146.190:43491)
0812 14:47:38.325020 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.325022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.325143 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:47:38.325184 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.325185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.325193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.325292 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.325739 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.326196 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.326866 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.327067 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.327117 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.329201 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.329203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.329203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.329231 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.330653 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.331014 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.331016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.331016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.331092 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.331289 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.331290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.331291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.331401 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.331506 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.331617 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.331696 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:47:38.333379 28376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:38.335137 28376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:38.341830 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.337823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.13.146.129:44679)
0812 14:47:38.338264 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.338269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.338280 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:38.338364 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.338366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.338384 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.338456 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.338665 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339346 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.340709 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341347 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341446 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.341510 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.341683 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.341724 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":95,"threads_started":1}
I20260812 14:47:38.341832 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.337495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:45961 (local address 127.0.0.1:45736)
0812 14:47:38.337667 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.337710 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.337809 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.338019 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.338022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.338024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.338099 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.338101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339301 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.339302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.340563 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.340566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341521 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.341609 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.341619 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.341652 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.341738 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":5,"thread_start_us":56,"threads_started":1}
I20260812 14:47:38.341837 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.337726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40393 (local address 127.0.0.1:50870)
0812 14:47:38.338354 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.338365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.338430 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.338600 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.338601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.338603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.338646 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.338647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339667 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.339668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.340582 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.340585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341556 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.341609 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.341617 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.341662 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.341752 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":41,"thread_start_us":69,"threads_started":1}
I20260812 14:47:38.342156 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.337535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.0.0.1:50258)
0812 14:47:38.337846 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.337855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.337904 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.338524 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.338525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.338526 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.338587 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.338588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339434 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.339435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.340637 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.340639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341427 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.341564 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.341576 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.341664 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.341873 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":175,"thread_start_us":91,"threads_started":1}
I20260812 14:47:38.342303 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.337590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.13.146.131:45961)
0812 14:47:38.337785 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.337790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.337804 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:38.337909 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.337911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.337929 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.338015 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.338166 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339213 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.340648 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341415 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341985 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.342081 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.342107 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.342145 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260812 14:47:38.342480 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.338108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.13.146.130:40393)
0812 14:47:38.338305 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.338309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.338447 (+ 138us) server_negotiation.cc:407] Connection header received
0812 14:47:38.338494 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.338497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.338511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.338593 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.338700 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.339581 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.340707 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.341412 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.341710 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.341777 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.341805 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.341844 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260812 14:47:38.343089 29225 tablet_service.cc:1467] Processing CreateTablet for tablet 0c173d122c5b44c2adb13836749cb96b (DEFAULT_TABLE table=test-table [id=a2f66399b23642869ff619e525971e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.343087 28656 tablet_service.cc:1467] Processing CreateTablet for tablet 0c173d122c5b44c2adb13836749cb96b (DEFAULT_TABLE table=test-table [id=a2f66399b23642869ff619e525971e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.343439 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c173d122c5b44c2adb13836749cb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.343446 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c173d122c5b44c2adb13836749cb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.345952 28931 tablet_service.cc:1467] Processing CreateTablet for tablet 0c173d122c5b44c2adb13836749cb96b (DEFAULT_TABLE table=test-table [id=a2f66399b23642869ff619e525971e22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.346313 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c173d122c5b44c2adb13836749cb96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.346968 29403 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Bootstrap starting.
I20260812 14:47:38.347630 29403 tablet_bootstrap.cc:654] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.347934 29403 log.cc:824] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:38.348692 29403 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: No bootstrap required, opened a new log
I20260812 14:47:38.348783 29403 ts_tablet_manager.cc:1397] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.349817 29410 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Bootstrap starting.
I20260812 14:47:38.349890 29402 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Bootstrap starting.
I20260812 14:47:38.350416 29402 tablet_bootstrap.cc:654] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.350488 29410 tablet_bootstrap.cc:654] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.350713 29402 log.cc:824] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:38.350831 29410 log.cc:824] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:38.350893 29403 raft_consensus.cc:348] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.351081 29403 raft_consensus.cc:374] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.351156 29403 raft_consensus.cc:729] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.351277 29403 consensus_queue.cc:260] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.351487 29402 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: No bootstrap required, opened a new log
I20260812 14:47:38.351560 29402 ts_tablet_manager.cc:1397] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.351614 28785 heartbeater.cc:503] Master 127.13.146.190:43491 was elected leader, sending a full tablet report...
I20260812 14:47:38.351825 29410 tablet_bootstrap.cc:492] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: No bootstrap required, opened a new log
I20260812 14:47:38.351559 29403 ts_tablet_manager.cc:1428] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:38.351894 29410 ts_tablet_manager.cc:1397] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.353588 29402 raft_consensus.cc:348] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.353732 29402 raft_consensus.cc:374] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.353793 29402 raft_consensus.cc:729] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e981b286bd8442fa9503162e88328893, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.353904 29402 consensus_queue.cc:260] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.353823 29410 raft_consensus.cc:348] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.353974 29410 raft_consensus.cc:374] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.354035 29410 raft_consensus.cc:729] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.354171 29402 ts_tablet_manager.cc:1428] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:38.354223 29359 heartbeater.cc:503] Master 127.13.146.190:43491 was elected leader, sending a full tablet report...
I20260812 14:47:38.354168 29410 consensus_queue.cc:260] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.354437 29410 ts_tablet_manager.cc:1428] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:47:38.354499 29072 heartbeater.cc:503] Master 127.13.146.190:43491 was elected leader, sending a full tablet report...
I20260812 14:47:38.377342 29415 raft_consensus.cc:482] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:38.377491 29415 raft_consensus.cc:504] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.377831 29415 leader_election.cc:288] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393)
I20260812 14:47:38.381618 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.378126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47181 (local address 127.13.146.129:44679)
0812 14:47:38.378206 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.378208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.378218 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:38.378248 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.378249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.378257 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.378320 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.378476 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.379252 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.380692 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.381258 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.381332 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.381340 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.381509 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.381546 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 14:47:38.381618 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.378263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51811 (local address 127.13.146.130:40393)
0812 14:47:38.378310 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.378312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.378387 (+ 75us) server_negotiation.cc:407] Connection header received
0812 14:47:38.378473 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.378474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.378483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.378538 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.378648 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.379330 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.380692 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.381258 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.381332 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.381340 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.381509 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.381546 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:38.381647 29140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.378016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.13.146.131:47181)
0812 14:47:38.378071 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.378083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.378129 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.378330 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.378331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.378333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.378407 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.378409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.379331 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.379332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.380598 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.380601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.381333 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.381334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.381433 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.381434 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.381494 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.381536 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":37}
I20260812 14:47:38.381649 29420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.378152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40393 (local address 127.13.146.131:51811)
0812 14:47:38.378332 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.378342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.378397 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.378551 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.378552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.378553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.378601 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.378602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.379399 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.379400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.380598 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.380601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.381334 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.381334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.381430 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.381434 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.381494 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.381536 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":8,"thread_start_us":112,"threads_started":1}
I20260812 14:47:38.382000 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c173d122c5b44c2adb13836749cb96b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" is_pre_election: true
I20260812 14:47:38.382167 28692 raft_consensus.cc:2454] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 0.
I20260812 14:47:38.382427 29126 leader_election.cc:302] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300ad4fcc4940768c5cf3b8c5882a2b, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:38.382455 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c173d122c5b44c2adb13836749cb96b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa" is_pre_election: true
I20260812 14:47:38.382624 28967 raft_consensus.cc:2454] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 0.
I20260812 14:47:38.382761 29415 raft_consensus.cc:2781] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:38.382846 29415 raft_consensus.cc:482] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:38.382884 29415 raft_consensus.cc:3037] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.383929 29415 raft_consensus.cc:504] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.384081 29415 leader_election.cc:288] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 election: Requested vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393)
I20260812 14:47:38.384223 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c173d122c5b44c2adb13836749cb96b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
I20260812 14:47:38.384258 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c173d122c5b44c2adb13836749cb96b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa"
I20260812 14:47:38.384328 28967 raft_consensus.cc:3037] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.384332 28692 raft_consensus.cc:3037] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.385087 28692 raft_consensus.cc:2454] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 1.
I20260812 14:47:38.385103 28967 raft_consensus.cc:2454] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 1.
I20260812 14:47:38.385293 29126 leader_election.cc:302] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300ad4fcc4940768c5cf3b8c5882a2b, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:38.385396 29415 raft_consensus.cc:2781] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:38.385587 29415 raft_consensus.cc:686] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 1 LEADER]: Becoming Leader. State: Replica: e981b286bd8442fa9503162e88328893, State: Running, Role: LEADER
I20260812 14:47:38.385686 29415 consensus_queue.cc:237] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } }
I20260812 14:47:38.386543 28376 catalog_manager.cc:5495] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 reported cstate change: term changed from 0 to 1, leader changed from <none> to e981b286bd8442fa9503162e88328893 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "e981b286bd8442fa9503162e88328893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } health_report { overall_health: UNKNOWN } } }
W20260812 14:47:38.388741 29073 tablet.cc:2369] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:38.398834 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.396518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.13.146.131:45961)
0812 14:47:38.396591 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.396593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.396607 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:38.396639 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.396640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.396656 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.396717 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:38.396844 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.397398 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.398161 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.398344 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.398399 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.398550 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.398644 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.398705 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.398764 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.398780 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:47:38.400648 28967 raft_consensus.cc:1261] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Refusing update from remote peer e981b286bd8442fa9503162e88328893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.400678 28692 raft_consensus.cc:1261] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Refusing update from remote peer e981b286bd8442fa9503162e88328893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.400991 29415 consensus_queue.cc:1035] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.401263 29422 consensus_queue.cc:1035] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.404383 29428 mvcc.cc:205] Tried to move back new op lower bound from 7317692655206211584 to 7317692655148683264. Current Snapshot: MvccSnapshot[applied={T|T < 7317692655206211584}]
I20260812 14:47:38.405050 29429 mvcc.cc:205] Tried to move back new op lower bound from 7317692655206211584 to 7317692655148683264. Current Snapshot: MvccSnapshot[applied={T|T < 7317692655206211584 or (T in {7317692655206211584})}]
I20260812 14:47:38.406090 28371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:38.406194 28371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:38.407003 28371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Sending DeleteTablet for 3 replicas of tablet 0c173d122c5b44c2adb13836749cb96b
I20260812 14:47:38.407371 29225 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c173d122c5b44c2adb13836749cb96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45736
I20260812 14:47:38.407634 29445 tablet_replica.cc:330] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: stopping tablet replica
I20260812 14:47:38.407619 28656 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c173d122c5b44c2adb13836749cb96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50258
I20260812 14:47:38.407611 28931 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c173d122c5b44c2adb13836749cb96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50870
I20260812 14:47:38.407737 29445 raft_consensus.cc:2229] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:38.407836 29445 raft_consensus.cc:2258] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.408051 28371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:38.408213 28371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:38.408241 29447 tablet_replica.cc:330] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: stopping tablet replica
I20260812 14:47:38.408344 29447 raft_consensus.cc:2229] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:38.408450 29447 raft_consensus.cc:2258] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.408885 29447 ts_tablet_manager.cc:1905] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.408919 29445 ts_tablet_manager.cc:1905] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.409787 29225 tablet_service.cc:1467] Processing CreateTablet for tablet 2573a750aaa84c9188606918da16aa52 (DEFAULT_TABLE table=test-table [id=20bafd022189426ca8529dde846de2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.409940 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2573a750aaa84c9188606918da16aa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.410669 29447 ts_tablet_manager.cc:1918] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:38.410739 29447 log.cc:1196] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/0c173d122c5b44c2adb13836749cb96b
I20260812 14:47:38.410772 29445 ts_tablet_manager.cc:1918] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:38.410837 29445 log.cc:1196] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/0c173d122c5b44c2adb13836749cb96b
I20260812 14:47:38.411010 29447 ts_tablet_manager.cc:1939] T 0c173d122c5b44c2adb13836749cb96b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting consensus metadata
I20260812 14:47:38.411137 29445 ts_tablet_manager.cc:1939] T 0c173d122c5b44c2adb13836749cb96b P e981b286bd8442fa9503162e88328893: Deleting consensus metadata
I20260812 14:47:38.411336 29402 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Bootstrap starting.
I20260812 14:47:38.411684 29402 tablet_bootstrap.cc:654] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.412055 28344 catalog_manager.cc:4841] TS 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679): tablet 0c173d122c5b44c2adb13836749cb96b (table test-table [id=a2f66399b23642869ff619e525971e22]) successfully deleted
I20260812 14:47:38.412125 28656 tablet_service.cc:1467] Processing CreateTablet for tablet 2573a750aaa84c9188606918da16aa52 (DEFAULT_TABLE table=test-table [id=20bafd022189426ca8529dde846de2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.412175 28931 tablet_service.cc:1467] Processing CreateTablet for tablet 2573a750aaa84c9188606918da16aa52 (DEFAULT_TABLE table=test-table [id=20bafd022189426ca8529dde846de2cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.412238 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2573a750aaa84c9188606918da16aa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.412279 29402 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: No bootstrap required, opened a new log
I20260812 14:47:38.412293 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2573a750aaa84c9188606918da16aa52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.412286 28347 catalog_manager.cc:4841] TS e981b286bd8442fa9503162e88328893 (127.13.146.131:45961): tablet 0c173d122c5b44c2adb13836749cb96b (table test-table [id=a2f66399b23642869ff619e525971e22]) successfully deleted
I20260812 14:47:38.412319 29402 ts_tablet_manager.cc:1397] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:38.412472 29402 raft_consensus.cc:348] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.412533 29402 raft_consensus.cc:374] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.412554 29402 raft_consensus.cc:729] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e981b286bd8442fa9503162e88328893, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.412593 29402 consensus_queue.cc:260] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.412673 29402 ts_tablet_manager.cc:1428] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.413555 29403 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Bootstrap starting.
I20260812 14:47:38.413748 29410 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Bootstrap starting.
I20260812 14:47:38.413978 29403 tablet_bootstrap.cc:654] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.414211 29410 tablet_bootstrap.cc:654] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.414716 29403 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: No bootstrap required, opened a new log
I20260812 14:47:38.414770 29403 ts_tablet_manager.cc:1397] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:38.414857 29410 tablet_bootstrap.cc:492] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: No bootstrap required, opened a new log
I20260812 14:47:38.414907 29410 ts_tablet_manager.cc:1397] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:38.414925 29403 raft_consensus.cc:348] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.414997 29403 raft_consensus.cc:374] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.415024 29403 raft_consensus.cc:729] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.415062 29403 consensus_queue.cc:260] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.415083 29410 raft_consensus.cc:348] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.415148 29403 ts_tablet_manager.cc:1428] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:38.415167 29410 raft_consensus.cc:374] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.415198 29410 raft_consensus.cc:729] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.415246 29410 consensus_queue.cc:260] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.415333 29410 ts_tablet_manager.cc:1428] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:38.421103 29452 tablet_replica.cc:330] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: stopping tablet replica
I20260812 14:47:38.421305 29452 raft_consensus.cc:2229] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:38.421450 29452 raft_consensus.cc:2258] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.421876 29452 ts_tablet_manager.cc:1905] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.423305 29452 ts_tablet_manager.cc:1918] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:38.423369 29452 log.cc:1196] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/0c173d122c5b44c2adb13836749cb96b
I20260812 14:47:38.423641 29452 ts_tablet_manager.cc:1939] T 0c173d122c5b44c2adb13836749cb96b P 751a62f5e6de40dd810c35500413e5aa: Deleting consensus metadata
I20260812 14:47:38.424104 28344 catalog_manager.cc:4841] TS 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393): tablet 0c173d122c5b44c2adb13836749cb96b (table test-table [id=a2f66399b23642869ff619e525971e22]) successfully deleted
W20260812 14:47:38.481261 28786 tablet.cc:2369] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:38.534138 29416 raft_consensus.cc:482] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:38.534252 29416 raft_consensus.cc:504] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.534644 29416 leader_election.cc:288] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:38.537660 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.534938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:39293 (local address 127.13.146.131:45961)
0812 14:47:38.534982 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.534984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.534994 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:38.535122 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.535123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.535131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.535194 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.535346 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.535806 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.536785 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.537304 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.537384 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.537393 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.537534 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.537587 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:38.537707 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.534881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:45961 (local address 127.13.146.130:39293)
0812 14:47:38.534945 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.534956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.535007 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.535200 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.535201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.535203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.535278 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.535279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.535888 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.535889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.536696 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.536698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.537381 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.537382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.537462 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.537465 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.537529 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.537579 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 14:47:38.537711 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.534936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57065 (local address 127.13.146.129:44679)
0812 14:47:38.534986 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.534988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.535187 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:47:38.535256 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.535258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.535267 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.535330 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.535523 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.535992 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.536942 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.537353 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.537430 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.537439 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.537588 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.537633 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:47:38.537850 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.534883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.13.146.130:57065)
0812 14:47:38.535087 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.535106 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.535194 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.535350 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.535351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.535353 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.535453 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.535455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.536079 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.536081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.536822 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.536824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.537431 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.537432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.537507 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.537511 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.537582 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.537623 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":75,"thread_start_us":73,"threads_started":1}
I20260812 14:47:38.538066 29261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2573a750aaa84c9188606918da16aa52" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893" is_pre_election: true
I20260812 14:47:38.538218 29261 raft_consensus.cc:2454] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 0.
I20260812 14:47:38.538466 28842 leader_election.cc:302] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:38.538446 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2573a750aaa84c9188606918da16aa52" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" is_pre_election: true
I20260812 14:47:38.538555 28692 raft_consensus.cc:2454] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 0.
I20260812 14:47:38.538640 29416 raft_consensus.cc:2781] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:38.538699 29416 raft_consensus.cc:482] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:38.538731 29416 raft_consensus.cc:3037] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.539430 29416 raft_consensus.cc:504] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.539585 29416 leader_election.cc:288] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 election: Requested vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:38.539755 29261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2573a750aaa84c9188606918da16aa52" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893"
I20260812 14:47:38.539783 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2573a750aaa84c9188606918da16aa52" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
I20260812 14:47:38.539844 29261 raft_consensus.cc:3037] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.539856 28692 raft_consensus.cc:3037] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.540541 28692 raft_consensus.cc:2454] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 1.
I20260812 14:47:38.540664 29261 raft_consensus.cc:2454] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 1.
I20260812 14:47:38.540723 28837 leader_election.cc:302] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, 8300ad4fcc4940768c5cf3b8c5882a2b; no voters:
I20260812 14:47:38.540899 29416 raft_consensus.cc:2781] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:38.541051 29416 raft_consensus.cc:686] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 1 LEADER]: Becoming Leader. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Running, Role: LEADER
I20260812 14:47:38.541149 29416 consensus_queue.cc:237] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
W20260812 14:47:38.541438 29361 tablet.cc:2369] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:47:38.541911 28371 catalog_manager.cc:5495] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 751a62f5e6de40dd810c35500413e5aa (127.13.146.130). New cstate: current_term: 1 leader_uuid: "751a62f5e6de40dd810c35500413e5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546057, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.190@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:38.601145 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.586049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:43491 (local address 127.0.0.1:39090)
0812 14:47:38.586281 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.586293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.586365 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.586504 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.586504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.586831 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:38.586880 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.586883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.587553 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.587554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.588085 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.588087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.588524 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.588526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.588580 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.596759 (+ 8179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:38.596766 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:38.598476 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:38.599983 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.599991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.599994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.600248 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.600446 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:38.600447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:38.600448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:38.600510 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:38.600809 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:38.600818 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:38.600867 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.600938 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.601032 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":117,"threads_started":1}
I20260812 14:47:38.601284 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.586126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.13.146.190:43491)
0812 14:47:38.586209 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.586211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.586362 (+ 151us) server_negotiation.cc:407] Connection header received
0812 14:47:38.586428 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.586428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.586439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.586482 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.586956 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.587484 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.588161 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.588352 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.588406 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.598600 (+ 10194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.598601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.598602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.598610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.599896 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.600323 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.600324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.600325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.600382 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.600573 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.600573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.600574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.600687 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.600741 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.601011 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.601087 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:47:38.603391 28371 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:39090:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:47:38.603601 28371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:38.605118 28656 tablet_service.cc:1467] Processing CreateTablet for tablet f457b1e855914bdd83483de8d14865f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3585863b841648478c1e0a76989c4f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:38.605118 29225 tablet_service.cc:1467] Processing CreateTablet for tablet f457b1e855914bdd83483de8d14865f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3585863b841648478c1e0a76989c4f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:38.605201 28931 tablet_service.cc:1467] Processing CreateTablet for tablet f457b1e855914bdd83483de8d14865f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3585863b841648478c1e0a76989c4f28]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:47:38.605316 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f457b1e855914bdd83483de8d14865f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.605316 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f457b1e855914bdd83483de8d14865f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.605398 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f457b1e855914bdd83483de8d14865f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.606590 29410 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: Bootstrap starting.
I20260812 14:47:38.606839 29402 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: Bootstrap starting.
I20260812 14:47:38.607079 29410 tablet_bootstrap.cc:654] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.607146 29402 tablet_bootstrap.cc:654] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.607512 29403 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: Bootstrap starting.
I20260812 14:47:38.607821 29402 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: No bootstrap required, opened a new log
I20260812 14:47:38.607872 29402 ts_tablet_manager.cc:1397] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:38.607954 29403 tablet_bootstrap.cc:654] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.607999 29402 raft_consensus.cc:348] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.608054 29402 raft_consensus.cc:374] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.608068 29402 raft_consensus.cc:729] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e981b286bd8442fa9503162e88328893, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.608095 29410 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: No bootstrap required, opened a new log
I20260812 14:47:38.608142 29410 ts_tablet_manager.cc:1397] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.608124 29402 consensus_queue.cc:260] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.608265 29402 ts_tablet_manager.cc:1428] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.608289 29440 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.608294 29410 raft_consensus.cc:348] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.608366 29410 raft_consensus.cc:374] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.608383 29410 raft_consensus.cc:729] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.608367 29440 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.608446 29410 consensus_queue.cc:260] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.608533 29410 ts_tablet_manager.cc:1428] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:38.608686 29416 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.608779 29416 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.608958 29403 tablet_bootstrap.cc:492] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: No bootstrap required, opened a new log
I20260812 14:47:38.609020 29403 ts_tablet_manager.cc:1397] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.609182 29403 raft_consensus.cc:348] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.609287 29403 raft_consensus.cc:374] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.609320 29403 raft_consensus.cc:729] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.609367 29403 consensus_queue.cc:260] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.609488 29403 ts_tablet_manager.cc:1428] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:38.609531 29414 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.609620 29414 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.817286 29414 raft_consensus.cc:482] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:38.817404 29414 raft_consensus.cc:504] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.817775 29414 leader_election.cc:288] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:38.820979 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.818128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:42211 (local address 127.13.146.131:45961)
0812 14:47:38.818205 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.818207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.818235 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:47:38.818317 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.818318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.818329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.818391 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.818623 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.819147 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820221 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.820610 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820666 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.820673 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.820853 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.820884 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:47:38.821017 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.817994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:45961 (local address 127.13.146.129:42211)
0812 14:47:38.818149 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.818164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.818235 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.818432 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.818433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.818436 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.818544 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.818547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.819239 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.819241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.820093 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.820097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.820682 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.820759 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.820764 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.820849 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.820901 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260812 14:47:38.821251 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.818089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:45123 (local address 127.13.146.130:40393)
0812 14:47:38.818196 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.818198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.818386 (+ 188us) server_negotiation.cc:407] Connection header received
0812 14:47:38.818433 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.818434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.818445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.818520 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:47:38.818701 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.819177 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820277 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.820856 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820929 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.820937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.821115 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.821161 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 14:47:38.821326 29261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f457b1e855914bdd83483de8d14865f4" candidate_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893" is_pre_election: true
I20260812 14:47:38.821466 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.818001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40393 (local address 127.13.146.129:45123)
0812 14:47:38.818264 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.818288 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.818360 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.818523 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.818524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.818525 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:47:38.818605 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.818614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.819298 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.819299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.820167 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.820170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.820939 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.820940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.821011 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.821015 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.821329 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.821377 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":38,"thread_start_us":69,"threads_started":1}
I20260812 14:47:38.821457 29261 raft_consensus.cc:2454] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300ad4fcc4940768c5cf3b8c5882a2b in term 0.
I20260812 14:47:38.821750 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f457b1e855914bdd83483de8d14865f4" candidate_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa" is_pre_election: true
I20260812 14:47:38.821857 28967 raft_consensus.cc:2454] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8300ad4fcc4940768c5cf3b8c5882a2b in term 0.
I20260812 14:47:38.821830 28548 leader_election.cc:302] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8300ad4fcc4940768c5cf3b8c5882a2b, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:38.821995 29414 raft_consensus.cc:2781] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:38.822036 29414 raft_consensus.cc:482] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:38.822052 29414 raft_consensus.cc:3037] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.822801 29414 raft_consensus.cc:504] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.822968 29414 leader_election.cc:288] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [CANDIDATE]: Term 1 election: Requested vote from peers 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:38.823124 29261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f457b1e855914bdd83483de8d14865f4" candidate_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893"
I20260812 14:47:38.823145 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f457b1e855914bdd83483de8d14865f4" candidate_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa"
I20260812 14:47:38.823221 29261 raft_consensus.cc:3037] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.823256 28967 raft_consensus.cc:3037] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:38.824026 28967 raft_consensus.cc:2454] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300ad4fcc4940768c5cf3b8c5882a2b in term 1.
I20260812 14:47:38.824026 29261 raft_consensus.cc:2454] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8300ad4fcc4940768c5cf3b8c5882a2b in term 1.
I20260812 14:47:38.824244 28546 leader_election.cc:302] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, 8300ad4fcc4940768c5cf3b8c5882a2b; no voters:
I20260812 14:47:38.824358 29414 raft_consensus.cc:2781] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:38.824488 29414 raft_consensus.cc:686] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 LEADER]: Becoming Leader. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Running, Role: LEADER
I20260812 14:47:38.824558 29414 consensus_queue.cc:237] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.824954 29485 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: TxnStatusTablet state changed. Reason: New leader 8300ad4fcc4940768c5cf3b8c5882a2b. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.825156 29485 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:38.825443 29489 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:47:38.825472 28371 catalog_manager.cc:5495] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129). New cstate: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:38.825573 29489 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:47:38.837828 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.834188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.0.0.1:50268)
0812 14:47:38.834336 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.834349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.834400 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.834699 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.834700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.834701 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:38.834756 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.834759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.835696 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.835697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.836641 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.836645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.837152 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.837154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.837250 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.837269 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.837600 (+ 331us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.837621 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.837664 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.837720 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 14:47:38.838099 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.834364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.13.146.129:44679)
0812 14:47:38.834491 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.834493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.834501 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:38.834535 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.834536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.834572 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.834643 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:38.834833 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.835576 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.836754 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.837051 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.837111 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.837381 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.837508 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.837574 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.838012 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.838034 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 14:47:38.841382 28967 raft_consensus.cc:1261] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Refusing update from remote peer 8300ad4fcc4940768c5cf3b8c5882a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.841449 29261 raft_consensus.cc:1261] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Refusing update from remote peer 8300ad4fcc4940768c5cf3b8c5882a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.841770 29414 consensus_queue.cc:1035] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.841913 29485 consensus_queue.cc:1035] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.842613 29440 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: TxnStatusTablet state changed. Reason: New leader 8300ad4fcc4940768c5cf3b8c5882a2b. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.842734 29440 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.842741 29416 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: TxnStatusTablet state changed. Reason: New leader 8300ad4fcc4940768c5cf3b8c5882a2b. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.842828 29416 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.843868 29414 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.843973 29414 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:38.844195 29440 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.844295 29440 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P e981b286bd8442fa9503162e88328893: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.845284 29485 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.845393 29485 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 8300ad4fcc4940768c5cf3b8c5882a2b: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:47:38.846103 29416 tablet_replica.cc:439] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } } }
I20260812 14:47:38.846213 29416 tablet_replica.cc:442] T f457b1e855914bdd83483de8d14865f4 P 751a62f5e6de40dd810c35500413e5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:47:38.849098 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.845736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.13.146.130:40393)
0812 14:47:38.845782 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.845784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.845792 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:38.845831 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.845832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.845854 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.845929 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:38.846114 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.846893 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.848005 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.848316 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.848392 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.848624 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.848775 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.848857 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.848972 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.849008 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:47:38.855324 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.852007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.13.146.130:51269)
0812 14:47:38.852171 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.852183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.852241 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.852411 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.852412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.852413 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:38.852505 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.852507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.853147 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.853148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.854251 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.854254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.854832 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.854834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.854902 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.854918 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.855118 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.855120 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.855178 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.855225 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 14:47:38.855306 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.852171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:51269 (local address 127.13.146.129:44679)
0812 14:47:38.852243 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.852245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.852256 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:38.852331 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.852332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.852344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.852408 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:38.852577 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.853057 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.854357 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.854716 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.854798 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.855015 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.855056 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.855112 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.855183 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.855205 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:47:38.862164 29261 raft_consensus.cc:1261] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Refusing update from remote peer 751a62f5e6de40dd810c35500413e5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.862317 28692 raft_consensus.cc:1261] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Refusing update from remote peer 751a62f5e6de40dd810c35500413e5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:38.862445 29416 consensus_queue.cc:1035] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.862587 29464 consensus_queue.cc:1035] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:38.864532 29426 mvcc.cc:205] Tried to move back new op lower bound from 7317692657097560064 to 7317692655785246720. Current Snapshot: MvccSnapshot[applied={T|T < 7317692657097560064}]
I20260812 14:47:38.873764 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.870419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:34397 (local address 127.13.146.130:40393)
0812 14:47:38.870468 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.870470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.870484 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:38.870587 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.870595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.870608 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.870664 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:38.870837 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.871623 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.872813 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.873122 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.873165 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.873460 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.873497 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.873526 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.873652 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.873676 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:47:38.874251 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.870248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40393 (local address 127.13.146.129:34397)
0812 14:47:38.870431 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:38.870444 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:38.870501 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:38.870677 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:38.870678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:38.870680 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:47:38.870751 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.870753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.871729 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.871730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.872705 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:38.872708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.873273 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.873274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:38.873342 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.873362 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:47:38.873570 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:47:38.873572 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:38.874094 (+ 522us) client_negotiation.cc:240] Negotiation successful
0812 14:47:38.874146 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260812 14:47:38.885308 28376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:38.885421 28376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:38.886094 28376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Sending DeleteTablet for 3 replicas of tablet 2573a750aaa84c9188606918da16aa52
I20260812 14:47:38.886394 29225 tablet_service.cc:1514] Processing DeleteTablet for tablet 2573a750aaa84c9188606918da16aa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45736
I20260812 14:47:38.886565 29445 tablet_replica.cc:330] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: stopping tablet replica
I20260812 14:47:38.886569 28656 tablet_service.cc:1514] Processing DeleteTablet for tablet 2573a750aaa84c9188606918da16aa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50258
I20260812 14:47:38.886605 28931 tablet_service.cc:1514] Processing DeleteTablet for tablet 2573a750aaa84c9188606918da16aa52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:38 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50870
I20260812 14:47:38.886643 29445 raft_consensus.cc:2229] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:38.886664 29447 tablet_replica.cc:330] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: stopping tablet replica
I20260812 14:47:38.886687 29445 raft_consensus.cc:2258] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.886734 29447 raft_consensus.cc:2229] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:38.886775 29447 raft_consensus.cc:2258] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.886768 29452 tablet_replica.cc:330] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: stopping tablet replica
I20260812 14:47:38.886831 29452 raft_consensus.cc:2229] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:38.886936 29452 raft_consensus.cc:2258] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:38.887250 29447 ts_tablet_manager.cc:1905] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.887421 29445 ts_tablet_manager.cc:1905] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.888999 29447 ts_tablet_manager.cc:1918] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:38.889057 29447 log.cc:1196] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/2573a750aaa84c9188606918da16aa52
I20260812 14:47:38.889181 29445 ts_tablet_manager.cc:1918] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:38.889257 29445 log.cc:1196] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/2573a750aaa84c9188606918da16aa52
I20260812 14:47:38.889321 29447 ts_tablet_manager.cc:1939] T 2573a750aaa84c9188606918da16aa52 P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting consensus metadata
I20260812 14:47:38.889480 29445 ts_tablet_manager.cc:1939] T 2573a750aaa84c9188606918da16aa52 P e981b286bd8442fa9503162e88328893: Deleting consensus metadata
I20260812 14:47:38.890023 28344 catalog_manager.cc:4841] TS 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679): tablet 2573a750aaa84c9188606918da16aa52 (table test-table [id=20bafd022189426ca8529dde846de2cd]) successfully deleted
I20260812 14:47:38.890058 28347 catalog_manager.cc:4841] TS e981b286bd8442fa9503162e88328893 (127.13.146.131:45961): tablet 2573a750aaa84c9188606918da16aa52 (table test-table [id=20bafd022189426ca8529dde846de2cd]) successfully deleted
I20260812 14:47:38.893448 29452 ts_tablet_manager.cc:1905] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:38.895187 29452 ts_tablet_manager.cc:1918] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:38.895269 29452 log.cc:1196] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/2573a750aaa84c9188606918da16aa52
I20260812 14:47:38.895479 29452 ts_tablet_manager.cc:1939] T 2573a750aaa84c9188606918da16aa52 P 751a62f5e6de40dd810c35500413e5aa: Deleting consensus metadata
I20260812 14:47:38.895859 28344 catalog_manager.cc:4841] TS 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393): tablet 2573a750aaa84c9188606918da16aa52 (table test-table [id=20bafd022189426ca8529dde846de2cd]) successfully deleted
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:47:38.933547 28472 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:38.925892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.13.146.190:43491)
0812 14:47:38.925949 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.925951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.926108 (+ 157us) server_negotiation.cc:407] Connection header received
0812 14:47:38.926168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.926169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.926181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.926252 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.927021 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.927823 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.929381 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.929666 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.929738 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.933478 (+ 3740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39102: BlockingRecv error: recv got EOF from 127.0.0.1:39102 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 14:47:38 dist-test-slave-8hhm krb5kdc[28268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM
I20260812 14:47:38.943658 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:38.934484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.13.146.190:43491)
0812 14:47:38.934525 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:38.934527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:38.934651 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:47:38.934759 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:38.934761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:38.934772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:38.934804 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:38.935467 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.936193 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.937392 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:38.937689 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:38.937730 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:38.940570 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:38.940572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:38.940572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:38.940580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:38.942352 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.942767 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.942769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.942769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.942833 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:38.943068 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:38.943069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:38.943069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:38.943213 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:38.943310 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:38.943473 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 14:47:38.943555 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:47:38.945442 28376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:47:38.945629 28376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:38.947103 29225 tablet_service.cc:1467] Processing CreateTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b (DEFAULT_TABLE table=test-table [id=d1ecdc569f264d34af49f6fd719b3426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.947196 28656 tablet_service.cc:1467] Processing CreateTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b (DEFAULT_TABLE table=test-table [id=d1ecdc569f264d34af49f6fd719b3426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.947265 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23e400cadfd249389e712e9c0fe8cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.947348 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23e400cadfd249389e712e9c0fe8cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.948622 29402 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Bootstrap starting.
I20260812 14:47:38.948776 28931 tablet_service.cc:1467] Processing CreateTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b (DEFAULT_TABLE table=test-table [id=d1ecdc569f264d34af49f6fd719b3426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:38.948905 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23e400cadfd249389e712e9c0fe8cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:38.948993 29402 tablet_bootstrap.cc:654] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.949287 29403 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Bootstrap starting.
I20260812 14:47:38.949733 29403 tablet_bootstrap.cc:654] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.950017 29402 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: No bootstrap required, opened a new log
I20260812 14:47:38.950062 29402 ts_tablet_manager.cc:1397] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:38.950214 29402 raft_consensus.cc:348] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.950287 29402 raft_consensus.cc:374] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.950309 29402 raft_consensus.cc:729] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e981b286bd8442fa9503162e88328893, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.950350 29402 consensus_queue.cc:260] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.950429 29402 ts_tablet_manager.cc:1428] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.951267 29403 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: No bootstrap required, opened a new log
I20260812 14:47:38.951331 29403 ts_tablet_manager.cc:1397] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:47:38.951474 29403 raft_consensus.cc:348] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.951534 29403 raft_consensus.cc:374] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.951553 29403 raft_consensus.cc:729] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.951589 29403 consensus_queue.cc:260] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.951678 29403 ts_tablet_manager.cc:1428] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:38.951761 29410 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Bootstrap starting.
I20260812 14:47:38.952289 29410 tablet_bootstrap.cc:654] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:38.952939 29410 tablet_bootstrap.cc:492] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: No bootstrap required, opened a new log
I20260812 14:47:38.952996 29410 ts_tablet_manager.cc:1397] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:38.953145 29410 raft_consensus.cc:348] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.953246 29410 raft_consensus.cc:374] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:38.953270 29410 raft_consensus.cc:729] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Initialized, Role: FOLLOWER
I20260812 14:47:38.953310 29410 consensus_queue.cc:260] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:38.953392 29410 ts_tablet_manager.cc:1428] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:39.089243 29440 raft_consensus.cc:482] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:39.089361 29440 raft_consensus.cc:504] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.089516 29440 leader_election.cc:288] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393)
I20260812 14:47:39.089799 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23e400cadfd249389e712e9c0fe8cb8b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" is_pre_election: true
I20260812 14:47:39.089821 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23e400cadfd249389e712e9c0fe8cb8b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa" is_pre_election: true
I20260812 14:47:39.089920 28967 raft_consensus.cc:2454] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 0.
I20260812 14:47:39.089926 28692 raft_consensus.cc:2454] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 0.
I20260812 14:47:39.090124 29126 leader_election.cc:302] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:39.090235 29440 raft_consensus.cc:2781] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:39.090281 29440 raft_consensus.cc:482] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:39.090306 29440 raft_consensus.cc:3037] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.091044 29440 raft_consensus.cc:504] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.091190 29440 leader_election.cc:288] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 election: Requested vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393)
I20260812 14:47:39.091383 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23e400cadfd249389e712e9c0fe8cb8b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
I20260812 14:47:39.091409 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23e400cadfd249389e712e9c0fe8cb8b" candidate_uuid: "e981b286bd8442fa9503162e88328893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "751a62f5e6de40dd810c35500413e5aa"
I20260812 14:47:39.091477 28967 raft_consensus.cc:3037] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.091495 28692 raft_consensus.cc:3037] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.092125 28967 raft_consensus.cc:2454] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 1.
I20260812 14:47:39.092207 28692 raft_consensus.cc:2454] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e981b286bd8442fa9503162e88328893 in term 1.
I20260812 14:47:39.092310 29126 leader_election.cc:302] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:39.092410 29440 raft_consensus.cc:2781] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:39.092526 29440 raft_consensus.cc:686] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 1 LEADER]: Becoming Leader. State: Replica: e981b286bd8442fa9503162e88328893, State: Running, Role: LEADER
I20260812 14:47:39.092573 29440 consensus_queue.cc:237] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.093169 28371 catalog_manager.cc:5495] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 reported cstate change: term changed from 0 to 1, leader changed from <none> to e981b286bd8442fa9503162e88328893 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "e981b286bd8442fa9503162e88328893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } health_report { overall_health: HEALTHY } } }
I20260812 14:47:39.098065 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:39.095961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.13.146.131:45961)
0812 14:47:39.096026 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:39.096028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:39.096040 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:39.096088 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:39.096089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:39.096098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:39.096161 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:39.096274 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:39.096744 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:39.097550 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:39.097735 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:39.097785 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:39.097898 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:39.097924 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:39.097959 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:39.097996 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 14:47:39.098013 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:47:39.098807 28967 raft_consensus.cc:1261] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Refusing update from remote peer e981b286bd8442fa9503162e88328893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:39.098842 28692 raft_consensus.cc:1261] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Refusing update from remote peer e981b286bd8442fa9503162e88328893: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:39.099089 29556 consensus_queue.cc:1035] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:39.099182 29440 consensus_queue.cc:1035] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:39.101825 28376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:39.101917 28376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:39.102453 28376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Sending DeleteTablet for 3 replicas of tablet 23e400cadfd249389e712e9c0fe8cb8b
I20260812 14:47:39.102718 29225 tablet_service.cc:1514] Processing DeleteTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45736
I20260812 14:47:39.102756 28656 tablet_service.cc:1514] Processing DeleteTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50258
I20260812 14:47:39.102774 28931 tablet_service.cc:1514] Processing DeleteTablet for tablet 23e400cadfd249389e712e9c0fe8cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50870
I20260812 14:47:39.102826 29445 tablet_replica.cc:330] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: stopping tablet replica
I20260812 14:47:39.102844 29447 tablet_replica.cc:330] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: stopping tablet replica
I20260812 14:47:39.102861 29452 tablet_replica.cc:330] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: stopping tablet replica
I20260812 14:47:39.102883 29447 raft_consensus.cc:2229] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:39.102890 29445 raft_consensus.cc:2229] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:39.102903 29452 raft_consensus.cc:2229] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:39.102934 29447 raft_consensus.cc:2258] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.102943 29445 raft_consensus.cc:2258] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.102944 29452 raft_consensus.cc:2258] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.103276 29447 ts_tablet_manager.cc:1905] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:39.103223 28376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:47:39.103322 29445 ts_tablet_manager.cc:1905] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 14:47:39.103384 28376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:47:39.103824 29452 ts_tablet_manager.cc:1905] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:39.104624 28931 tablet_service.cc:1467] Processing CreateTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 (DEFAULT_TABLE table=test-table [id=1e3492c36bc84ee88b1035e142180157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:39.104741 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:39.104954 29447 ts_tablet_manager.cc:1918] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:39.105010 29447 log.cc:1196] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/23e400cadfd249389e712e9c0fe8cb8b
I20260812 14:47:39.105280 29447 ts_tablet_manager.cc:1939] T 23e400cadfd249389e712e9c0fe8cb8b P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting consensus metadata
I20260812 14:47:39.105576 29445 ts_tablet_manager.cc:1918] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:39.105633 29445 log.cc:1196] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/23e400cadfd249389e712e9c0fe8cb8b
I20260812 14:47:39.105712 29452 ts_tablet_manager.cc:1918] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:47:39.105764 29452 log.cc:1196] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/23e400cadfd249389e712e9c0fe8cb8b
I20260812 14:47:39.105799 29410 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Bootstrap starting.
I20260812 14:47:39.105857 29445 ts_tablet_manager.cc:1939] T 23e400cadfd249389e712e9c0fe8cb8b P e981b286bd8442fa9503162e88328893: Deleting consensus metadata
I20260812 14:47:39.105974 29452 ts_tablet_manager.cc:1939] T 23e400cadfd249389e712e9c0fe8cb8b P 751a62f5e6de40dd810c35500413e5aa: Deleting consensus metadata
I20260812 14:47:39.105974 28344 catalog_manager.cc:4841] TS 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679): tablet 23e400cadfd249389e712e9c0fe8cb8b (table test-table [id=d1ecdc569f264d34af49f6fd719b3426]) successfully deleted
I20260812 14:47:39.106142 28656 tablet_service.cc:1467] Processing CreateTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 (DEFAULT_TABLE table=test-table [id=1e3492c36bc84ee88b1035e142180157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:39.106199 29410 tablet_bootstrap.cc:654] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:39.106248 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:39.106356 28347 catalog_manager.cc:4841] TS e981b286bd8442fa9503162e88328893 (127.13.146.131:45961): tablet 23e400cadfd249389e712e9c0fe8cb8b (table test-table [id=d1ecdc569f264d34af49f6fd719b3426]) successfully deleted
I20260812 14:47:39.106427 28344 catalog_manager.cc:4841] TS 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393): tablet 23e400cadfd249389e712e9c0fe8cb8b (table test-table [id=d1ecdc569f264d34af49f6fd719b3426]) successfully deleted
I20260812 14:47:39.106503 29225 tablet_service.cc:1467] Processing CreateTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 (DEFAULT_TABLE table=test-table [id=1e3492c36bc84ee88b1035e142180157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:47:39.106611 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:39.107178 29403 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: Bootstrap starting.
I20260812 14:47:39.107466 29403 tablet_bootstrap.cc:654] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:39.107707 29402 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Bootstrap starting.
I20260812 14:47:39.108026 29402 tablet_bootstrap.cc:654] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:39.108132 29403 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: No bootstrap required, opened a new log
I20260812 14:47:39.108186 29403 ts_tablet_manager.cc:1397] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:39.108279 29410 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: No bootstrap required, opened a new log
I20260812 14:47:39.108338 29410 ts_tablet_manager.cc:1397] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:47:39.108301 29403 raft_consensus.cc:348] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.108407 29403 raft_consensus.cc:374] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:39.108434 29403 raft_consensus.cc:729] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8300ad4fcc4940768c5cf3b8c5882a2b, State: Initialized, Role: FOLLOWER
I20260812 14:47:39.108484 29403 consensus_queue.cc:260] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.108459 29410 raft_consensus.cc:348] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.108536 29410 raft_consensus.cc:374] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:39.108573 29410 raft_consensus.cc:729] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Initialized, Role: FOLLOWER
I20260812 14:47:39.108582 29403 ts_tablet_manager.cc:1428] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:39.108618 29410 consensus_queue.cc:260] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.108723 29410 ts_tablet_manager.cc:1428] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:39.108979 29402 tablet_bootstrap.cc:492] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: No bootstrap required, opened a new log
I20260812 14:47:39.109021 29402 ts_tablet_manager.cc:1397] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:39.109133 29402 raft_consensus.cc:348] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.109200 29402 raft_consensus.cc:374] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:39.109248 29402 raft_consensus.cc:729] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e981b286bd8442fa9503162e88328893, State: Initialized, Role: FOLLOWER
I20260812 14:47:39.109293 29402 consensus_queue.cc:260] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.109398 29402 ts_tablet_manager.cc:1428] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:39.174168 29464 raft_consensus.cc:482] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:47:39.174296 29464 raft_consensus.cc:504] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.174494 29464 leader_election.cc:288] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:39.174741 29258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fdeda2f21f45b5bd7ee9b6b804bc79" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893" is_pre_election: true
I20260812 14:47:39.174741 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fdeda2f21f45b5bd7ee9b6b804bc79" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" is_pre_election: true
I20260812 14:47:39.174855 29258 raft_consensus.cc:2454] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 0.
I20260812 14:47:39.174853 28692 raft_consensus.cc:2454] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 0.
I20260812 14:47:39.175065 28842 leader_election.cc:302] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:39.175194 29464 raft_consensus.cc:2781] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:47:39.175269 29464 raft_consensus.cc:482] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:47:39.175285 29464 raft_consensus.cc:3037] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.176079 29464 raft_consensus.cc:504] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.176221 29464 leader_election.cc:288] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 election: Requested vote from peers 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679), e981b286bd8442fa9503162e88328893 (127.13.146.131:45961)
I20260812 14:47:39.176388 29258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fdeda2f21f45b5bd7ee9b6b804bc79" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e981b286bd8442fa9503162e88328893"
I20260812 14:47:39.176385 28692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fdeda2f21f45b5bd7ee9b6b804bc79" candidate_uuid: "751a62f5e6de40dd810c35500413e5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b"
I20260812 14:47:39.176460 29258 raft_consensus.cc:3037] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.176460 28692 raft_consensus.cc:3037] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:39.177276 28692 raft_consensus.cc:2454] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 1.
I20260812 14:47:39.177285 29258 raft_consensus.cc:2454] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 751a62f5e6de40dd810c35500413e5aa in term 1.
I20260812 14:47:39.177452 28842 leader_election.cc:302] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 751a62f5e6de40dd810c35500413e5aa, e981b286bd8442fa9503162e88328893; no voters:
I20260812 14:47:39.177549 29464 raft_consensus.cc:2781] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:39.177615 29464 raft_consensus.cc:686] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 1 LEADER]: Becoming Leader. State: Replica: 751a62f5e6de40dd810c35500413e5aa, State: Running, Role: LEADER
I20260812 14:47:39.177654 29464 consensus_queue.cc:237] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } }
I20260812 14:47:39.178264 28376 catalog_manager.cc:5495] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 751a62f5e6de40dd810c35500413e5aa (127.13.146.130). New cstate: current_term: 1 leader_uuid: "751a62f5e6de40dd810c35500413e5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "751a62f5e6de40dd810c35500413e5aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 } health_report { overall_health: UNKNOWN } } }
I20260812 14:47:39.185555 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:39.183592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.13.146.130:40393)
0812 14:47:39.183636 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:39.183637 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:39.183661 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:47:39.183715 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:39.183716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:39.183724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:39.183764 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:47:39.183847 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:39.184294 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:39.185002 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:39.185173 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:39.185268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:39.185393 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:47:39.185418 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:47:39.185453 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:39.185488 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:47:39.185505 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:47:39.187642 28692 raft_consensus.cc:1261] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Refusing update from remote peer 751a62f5e6de40dd810c35500413e5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:39.187675 29258 raft_consensus.cc:1261] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Refusing update from remote peer 751a62f5e6de40dd810c35500413e5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:47:39.187870 29464 consensus_queue.cc:1035] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8300ad4fcc4940768c5cf3b8c5882a2b" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:39.187954 29511 consensus_queue.cc:1035] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e981b286bd8442fa9503162e88328893" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 45961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:47:39.198068 28371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:39.198201 28371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:47:39.198804 28371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d535937cfb98413ba21d4e14a8c4eee9: Sending DeleteTablet for 3 replicas of tablet 21fdeda2f21f45b5bd7ee9b6b804bc79
I20260812 14:47:39.199079 29225 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45736
I20260812 14:47:39.199115 28931 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50870
I20260812 14:47:39.199115 28656 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:47:39 UTC) from {username='oryx', principal='oryx/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50258
I20260812 14:47:39.199180 29445 tablet_replica.cc:330] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: stopping tablet replica
I20260812 14:47:39.199219 29447 tablet_replica.cc:330] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: stopping tablet replica
I20260812 14:47:39.199234 29445 raft_consensus.cc:2229] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:39.199270 29445 raft_consensus.cc:2258] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.199281 29447 raft_consensus.cc:2229] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:47:39.199319 29447 raft_consensus.cc:2258] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.199399 29452 tablet_replica.cc:330] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: stopping tablet replica
I20260812 14:47:39.199460 29452 raft_consensus.cc:2229] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:47:39.199527 29452 raft_consensus.cc:2258] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:47:39.199568 29445 ts_tablet_manager.cc:1905] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:39.199676 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 28516
I20260812 14:47:39.199872 29452 ts_tablet_manager.cc:1905] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:39.200012 29447 ts_tablet_manager.cc:1905] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 8300ad4fcc4940768c5cf3b8c5882a2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:47:39.211851 29445 ts_tablet_manager.cc:1918] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:39.211939 29445 log.cc:1196] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/21fdeda2f21f45b5bd7ee9b6b804bc79
I20260812 14:47:39.212239 29445 ts_tablet_manager.cc:1939] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P e981b286bd8442fa9503162e88328893: Deleting consensus metadata
I20260812 14:47:39.212710 28347 catalog_manager.cc:4841] TS e981b286bd8442fa9503162e88328893 (127.13.146.131:45961): tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 (table test-table [id=1e3492c36bc84ee88b1035e142180157]) successfully deleted
I20260812 14:47:39.213398 29452 ts_tablet_manager.cc:1918] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:47:39.213477 29452 log.cc:1196] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/21fdeda2f21f45b5bd7ee9b6b804bc79
I20260812 14:47:39.213685 29452 ts_tablet_manager.cc:1939] T 21fdeda2f21f45b5bd7ee9b6b804bc79 P 751a62f5e6de40dd810c35500413e5aa: Deleting consensus metadata
I20260812 14:47:39.214037 28344 catalog_manager.cc:4841] TS 751a62f5e6de40dd810c35500413e5aa (127.13.146.130:40393): tablet 21fdeda2f21f45b5bd7ee9b6b804bc79 (table test-table [id=1e3492c36bc84ee88b1035e142180157]) successfully deleted
W20260812 14:47:39.219527 28344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:44679 (error 108)
I20260812 14:47:39.219849 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 28807
I20260812 14:47:39.227010 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:39.220003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44679 (local address 127.0.0.1:50270)
0812 14:47:39.226838 (+ 6835us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:39.226946 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6807,"thread_start_us":44,"threads_started":1}
W20260812 14:47:39.227833 28344 catalog_manager.cc:4584] TS 8300ad4fcc4940768c5cf3b8c5882a2b (127.13.146.129:44679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21fdeda2f21f45b5bd7ee9b6b804bc79: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44679: connect: Connection refused (error 111)
I20260812 14:47:39.233515 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 29091
I20260812 14:47:39.247296 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 28311
2026-08-12T14:47:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3063 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:39 dist-test-slave-8hhm krb5kdc[29581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:39 dist-test-slave-8hhm krb5kdc[29581](info): set up 2 sockets
Aug 12 14:47:39 dist-test-slave-8hhm krb5kdc[29581](info): commencing operation
krb5kdc: starting...
W20260812 14:47:40.575258 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:40 dist-test-slave-8hhm krb5kdc[29581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:40.704222 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46375
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:40.789640 29608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:40.789956 29608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:40.789999 29608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:40.791177 29608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:40.791214 29608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:40.791229 29608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:40.791241 29608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:40.791253 29608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:40.792714 29608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:40.792904 29608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:40.793089 29608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:40.797981 29614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:40.797963 29615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:40.798290 29608 server_base.cc:1034] running on GCE node
W20260812 14:47:40.797952 29617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:40.798599 29608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:40.798856 29608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:40.799999 29608 hybrid_clock.cc:648] HybridClock initialized: now 1786546060799987 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T14:47:40Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1552 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:47:40 dist-test-slave-8hhm krb5kdc[29625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:47:40 dist-test-slave-8hhm krb5kdc[29625](info): set up 2 sockets
Aug 12 14:47:40 dist-test-slave-8hhm krb5kdc[29625](info): commencing operation
krb5kdc: starting...
W20260812 14:47:42.137112 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:47:42.267561 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35081
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:35081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:42.352873 29649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:42.353137 29649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:42.353175 29649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:42.354338 29649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:42.354408 29649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:42.354431 29649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:42.354451 29649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:42.354480 29649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:42.355991 29649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:35081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:42.356283 29649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:42.356508 29649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:42.361451 29661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.361430 29659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.361423 29666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:42.361809 29649 server_base.cc:1034] running on GCE node
I20260812 14:47:42.362073 29649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:42.362380 29649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:42.363565 29649 hybrid_clock.cc:648] HybridClock initialized: now 1786546062363553 us; error 45 us; skew 500 ppm
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:42.366111 29649 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:42.366494 29649 webserver.cc:460] Webserver started at http://127.13.146.190:46667/ using document root <none> and password file <none>
I20260812 14:47:42.366760 29649 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:42.366827 29649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:42.366982 29649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:42.368060 29649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "094f8c03afb84dd3824583df59095c47"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "dd3a521e6e2a589c11162aedbe9bfaab"
server_key_iv: "c1311cde501b0d36e9f20a7c5bd17606"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.368433 29649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "094f8c03afb84dd3824583df59095c47"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "dd3a521e6e2a589c11162aedbe9bfaab"
server_key_iv: "c1311cde501b0d36e9f20a7c5bd17606"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.369891 29649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:47:42.370718 29675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.370903 29649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:42.371011 29649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "094f8c03afb84dd3824583df59095c47"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "dd3a521e6e2a589c11162aedbe9bfaab"
server_key_iv: "c1311cde501b0d36e9f20a7c5bd17606"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.371105 29649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:42.386845 29649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:42.388914 29649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:42.389088 29649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:42.393301 29649 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:35081
I20260812 14:47:42.393339 29781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:35081 every 8 connection(s)
I20260812 14:47:42.393728 29649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:42.394629 29782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:42.397058 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Bootstrap starting.
I20260812 14:47:42.397716 29782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:42.398020 29782 log.cc:824] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:42.398756 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: No bootstrap required, opened a new log
I20260812 14:47:42.400369 29782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:42.400511 29782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:42.400534 29782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094f8c03afb84dd3824583df59095c47, State: Initialized, Role: FOLLOWER
I20260812 14:47:42.400709 29782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:42.400776 29782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:42.400820 29782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:42.400869 29782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:42.401419 29782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:42.401533 29782 leader_election.cc:302] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 094f8c03afb84dd3824583df59095c47; no voters:
I20260812 14:47:42.401721 29782 leader_election.cc:288] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:42.401754 29789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:42.401965 29789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 LEADER]: Becoming Leader. State: Replica: 094f8c03afb84dd3824583df59095c47, State: Running, Role: LEADER
I20260812 14:47:42.401996 29782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:42.402071 29789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:42.402732 29790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "094f8c03afb84dd3824583df59095c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } } }
I20260812 14:47:42.402828 29790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:42.403067 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 29649
I20260812 14:47:42.403095 29802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:42.403180 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:42.403206 29791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 094f8c03afb84dd3824583df59095c47. Latest consensus state: current_term: 1 leader_uuid: "094f8c03afb84dd3824583df59095c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } } }
I20260812 14:47:42.403288 29791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:42.403363 13898 external_mini_cluster.cc:1428] Setting key f7107834440072b63b3c00c794b1d081
I20260812 14:47:42.403687 29802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:42.405597 29802 catalog_manager.cc:1347] Generated new cluster ID: 22ada8dc746c4a689937926a037e6e30
I20260812 14:47:42.405673 29802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:42.416951 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.403902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38102 (local address 127.13.146.190:35081)
0812 14:47:42.404327 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:42.404336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:42.404373 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:47:42.404492 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:42.404499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:42.404543 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:42.404683 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:42.405451 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.406827 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.407968 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.408591 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.409050 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.411817 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:42.411821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:42.411822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:42.411834 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:42.414757 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.415213 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.415215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.415216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.415342 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.415611 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.415612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.415613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.415932 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:42.416502 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:42.416580 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:47:42.416729 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":84,"threads_started":1}
I20260812 14:47:42.430555 29802 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:42.431104 29802 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:42.437484 29802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Generated new TSK 0
I20260812 14:47:42.437675 29802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:47:42.470624 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:42.556470 29828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:42.556734 29828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:42.556775 29828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:42.557996 29828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:42.558072 29828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:42.558122 29828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:42.559646 29828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:42.559931 29828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:42.560165 29828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:42.565397 29845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.565418 29841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.565444 29842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:42.565775 29828 server_base.cc:1034] running on GCE node
I20260812 14:47:42.566017 29828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:42.566272 29828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:42.567447 29828 hybrid_clock.cc:648] HybridClock initialized: now 1786546062567430 us; error 39 us; skew 500 ppm
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:42.570240 29828 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:47:42.570614 29828 webserver.cc:460] Webserver started at http://127.13.146.129:37241/ using document root <none> and password file <none>
I20260812 14:47:42.570804 29828 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:42.570845 29828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:42.570933 29828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:42.571940 29828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "b1a68e4ce4c347faae5a37f32e64fa24"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "a939970bbb5414e65745942e628358a7"
server_key_iv: "196794c1219811e0f3de5982096f8071"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.572254 29828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "b1a68e4ce4c347faae5a37f32e64fa24"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "a939970bbb5414e65745942e628358a7"
server_key_iv: "196794c1219811e0f3de5982096f8071"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.573587 29828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:42.574393 29856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.574575 29828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:42.574676 29828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "b1a68e4ce4c347faae5a37f32e64fa24"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "a939970bbb5414e65745942e628358a7"
server_key_iv: "196794c1219811e0f3de5982096f8071"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.574754 29828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:42.586184 29828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:42.588032 29828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:42.588205 29828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:42.588436 29828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:42.588747 29828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:42.588797 29828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.588840 29828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:42.588865 29828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.594782 29828 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:41267
I20260812 14:47:42.594856 30084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:41267 every 8 connection(s)
I20260812 14:47:42.595196 29828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:42.596840 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 29828
I20260812 14:47:42.596967 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:42.597199 13898 external_mini_cluster.cc:1428] Setting key 8313bd21917e3ecc7d6fbe0448a9728d
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:42.605767 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.596006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:41579 (local address 127.13.146.190:35081)
0812 14:47:42.596091 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:42.596093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:42.596307 (+ 214us) server_negotiation.cc:407] Connection header received
0812 14:47:42.596844 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:42.596845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:42.596861 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:42.596927 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:42.598245 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.598750 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.599741 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.599956 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.600027 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.602647 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:42.602648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:42.602648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:42.602655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:42.604137 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.604686 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.604687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.604687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.604740 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.605065 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.605065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.605066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.605168 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:42.605256 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:42.605617 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 14:47:42.605700 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:47:42.606055 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.595984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:41579)
0812 14:47:42.596188 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:42.596226 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:42.596735 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:42.597001 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:42.597005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:42.597847 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:42.598132 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.598139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.598843 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.598844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.599652 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.599654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.600036 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.600037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.600173 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.600725 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:42.600734 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:42.602525 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:42.604221 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.604230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.604232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.604597 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.604808 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.604809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.604809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.604978 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.605272 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:42.605280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:42.605458 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 14:47:42.605602 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 14:47:42.605794 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":117,"threads_started":1}
I20260812 14:47:42.606650 30087 heartbeater.cc:347] Connected to a master server at 127.13.146.190:35081
I20260812 14:47:42.606745 30087 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:42.606922 30087 heartbeater.cc:511] Master 127.13.146.190:35081 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
I20260812 14:47:42.607456 29712 ts_manager.cc:194] Registered new tserver with Master: b1a68e4ce4c347faae5a37f32e64fa24 (127.13.146.129:41267)
I20260812 14:47:42.608068 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:41579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:47:42.626600 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:42.712517 30096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:42.712725 30096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:42.712750 30096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:42.713958 30096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:42.714011 30096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:42.714064 30096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:42.715502 30096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:42.715746 30096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:42.715977 30096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:42.721061 30107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.721072 30108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:42.721091 30096 server_base.cc:1034] running on GCE node
W20260812 14:47:42.721091 30111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:42.721568 30096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:42.721840 30096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:42.722985 30096 hybrid_clock.cc:648] HybridClock initialized: now 1786546062722967 us; error 33 us; skew 500 ppm
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:42.725570 30096 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:47:42.726011 30096 webserver.cc:460] Webserver started at http://127.13.146.130:43445/ using document root <none> and password file <none>
I20260812 14:47:42.726243 30096 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:42.726289 30096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:42.726431 30096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:42.727516 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "67acd14ef44c44ae9a5d7ee1918e2877"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "ea05dedae6d69ccaa7fb3c9c3e664e86"
server_key_iv: "b46364e3965dde7ac246916400db9991"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.727878 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "67acd14ef44c44ae9a5d7ee1918e2877"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "ea05dedae6d69ccaa7fb3c9c3e664e86"
server_key_iv: "b46364e3965dde7ac246916400db9991"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.729398 30096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 14:47:42.730224 30126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.730391 30096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:42.730484 30096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "67acd14ef44c44ae9a5d7ee1918e2877"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "ea05dedae6d69ccaa7fb3c9c3e664e86"
server_key_iv: "b46364e3965dde7ac246916400db9991"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.730544 30096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:42.742467 30096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:42.744382 30096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:42.744498 30096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:42.744686 30096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:42.744936 30096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:42.744966 30096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.744992 30096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:42.745005 30096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.750712 30096 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:39049
I20260812 14:47:42.750782 30335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:39049 every 8 connection(s)
I20260812 14:47:42.751111 30096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:42.752549 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 30096
I20260812 14:47:42.752681 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:42.752902 13898 external_mini_cluster.cc:1428] Setting key c02ff4f0ccfcb6e08dd116b6144c64ac
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:42.762264 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.751851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:56597 (local address 127.13.146.190:35081)
0812 14:47:42.751918 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:42.751920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:42.752121 (+ 201us) server_negotiation.cc:407] Connection header received
0812 14:47:42.752624 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:42.752626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:42.752640 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:42.752721 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:42.754138 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.754629 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.755929 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.756255 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.756333 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.759095 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:42.759097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:42.759097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:42.759105 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:42.760627 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.761151 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.761153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.761153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.761270 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.761567 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.761568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.761568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.761682 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:42.761798 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:42.762127 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 14:47:42.762200 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:47:42.762573 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.751813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:56597)
0812 14:47:42.751993 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:42.752036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:42.752531 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:42.752874 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:42.752879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:42.753727 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:42.754008 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.754015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.754718 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.754719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.755841 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.755844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.756371 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.756373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.756482 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.757019 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:42.757028 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:42.758889 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:42.760717 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.760724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.760727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.761062 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.761347 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.761348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.761348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.761496 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.761785 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:42.761792 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:42.761966 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 14:47:42.762101 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 14:47:42.762300 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:47:42.763187 30337 heartbeater.cc:347] Connected to a master server at 127.13.146.190:35081
I20260812 14:47:42.763288 30337 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:42.763432 30337 heartbeater.cc:511] Master 127.13.146.190:35081 requested a full tablet report, sending...
I20260812 14:47:42.763852 29712 ts_manager.cc:194] Registered new tserver with Master: 67acd14ef44c44ae9a5d7ee1918e2877 (127.13.146.130:39049)
I20260812 14:47:42.764335 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:56597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:47:42.780119 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:42.863874 30348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:42.864138 30348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:42.864178 30348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:42.865489 30348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:42.865548 30348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:42.865584 30348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:42.867085 30348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:42.867338 30348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:42.867580 30348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:42.872807 30357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.872814 30359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:42.872782 30356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:42.873153 30348 server_base.cc:1034] running on GCE node
I20260812 14:47:42.873394 30348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:42.873656 30348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:42.874840 30348 hybrid_clock.cc:648] HybridClock initialized: now 1786546062874831 us; error 35 us; skew 500 ppm
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:42.877494 30348 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:47:42.878068 30348 webserver.cc:460] Webserver started at http://127.13.146.131:46407/ using document root <none> and password file <none>
I20260812 14:47:42.878324 30348 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:42.878391 30348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:42.878540 30348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:42.879660 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "a98a0287498e4a90af30bd62656947e9"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "b4d68fef0a1e94f048eb13663d6416c5"
server_key_iv: "52824459709615f486960f8159341fe1"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.880060 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "a98a0287498e4a90af30bd62656947e9"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "b4d68fef0a1e94f048eb13663d6416c5"
server_key_iv: "52824459709615f486960f8159341fe1"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.881583 30348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 14:47:42.882622 30373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.882825 30348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:42.882932 30348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "a98a0287498e4a90af30bd62656947e9"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "b4d68fef0a1e94f048eb13663d6416c5"
server_key_iv: "52824459709615f486960f8159341fe1"
server_key_version: "encryptionkey@0"
I20260812 14:47:42.883013 30348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:42.899930 30348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:42.902467 30348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:42.902616 30348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:42.902848 30348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:42.903164 30348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:42.903208 30348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.903254 30348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:42.903298 30348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:42.909164 30348 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41907
I20260812 14:47:42.909271 30601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41907 every 8 connection(s)
I20260812 14:47:42.909605 30348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:42.915621 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 30348
I20260812 14:47:42.915767 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
Aug 12 14:47:42 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546062, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:47:42.915987 13898 external_mini_cluster.cc:1428] Setting key 9efca5c52034beda62c1394c174e3cef
I20260812 14:47:42.919370 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.910320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:36195 (local address 127.13.146.190:35081)
0812 14:47:42.910378 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:42.910380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:42.910607 (+ 227us) server_negotiation.cc:407] Connection header received
0812 14:47:42.911152 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:42.911154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:42.911166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:42.911224 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:42.912225 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.912727 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.913806 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.913997 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.914085 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.916445 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:42.916447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:42.916447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:42.916456 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:42.917965 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.918392 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.918393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.918394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.918459 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:42.918706 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:42.918707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:42.918707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:42.918832 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:42.918908 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:42.919189 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 14:47:42.919282 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:47:42.919554 30609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:42.910307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:36195)
0812 14:47:42.910477 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:42.910512 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:42.911022 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:42.911300 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:42.911304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:42.911923 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:42.912116 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.912121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.912818 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.912819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.913717 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:42.913720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:42.914098 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:42.914099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:42.914230 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:42.914795 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:42.914805 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:42.916347 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:42.918070 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.918077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.918079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.918313 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.918526 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:47:42.918527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:47:42.918528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:47:42.918641 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:47:42.918910 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:42.918917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:42.919056 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 14:47:42.919162 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:47:42.919319 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260812 14:47:42.920083 30607 heartbeater.cc:347] Connected to a master server at 127.13.146.190:35081
I20260812 14:47:42.920195 30607 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:42.920388 30607 heartbeater.cc:511] Master 127.13.146.190:35081 requested a full tablet report, sending...
I20260812 14:47:42.920776 29712 ts_manager.cc:194] Registered new tserver with Master: a98a0287498e4a90af30bd62656947e9 (127.13.146.131:41907)
I20260812 14:47:42.921181 29712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:36195
I20260812 14:47:42.927897 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
Entry for principal oryx/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab.
WARNING: no policy specified for oryx/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
Entry for principal oryx/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab.
WARNING: no policy specified for oryx/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
Entry for principal oryx/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab.
I20260812 14:47:43.004400 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 29828
I20260812 14:47:43.022414 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 30096
I20260812 14:47:43.042261 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 30348
I20260812 14:47:43.049988 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 29649
I20260812 14:47:43.063827 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35081
--webserver_interface=127.13.146.190
--webserver_port=46667
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:35081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:43.147413 30618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:43.147619 30618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:43.147640 30618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:43.148957 30618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:43.149003 30618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:43.149016 30618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:43.149029 30618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:43.149073 30618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:43.150591 30618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:35081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=46667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:43.150852 30618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:43.151080 30618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:43.156162 30630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.156224 30618 server_base.cc:1034] running on GCE node
W20260812 14:47:43.156189 30627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:43.156189 30634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.156626 30618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:43.156929 30618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:43.158090 30618 hybrid_clock.cc:648] HybridClock initialized: now 1786546063158078 us; error 38 us; skew 500 ppm
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546063, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:43.160710 30618 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:47:43.161170 30618 webserver.cc:460] Webserver started at http://127.13.146.190:46667/ using document root <none> and password file <none>
I20260812 14:47:43.161459 30618 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:43.161546 30618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:43.162843 30618 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:43.163516 30641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.163684 30618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:43.163791 30618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "094f8c03afb84dd3824583df59095c47"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "dd3a521e6e2a589c11162aedbe9bfaab"
server_key_iv: "c1311cde501b0d36e9f20a7c5bd17606"
server_key_version: "encryptionkey@0"
I20260812 14:47:43.164143 30618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:43.181264 30618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:43.183681 30618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:43.183847 30618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:43.188025 30618 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:35081
I20260812 14:47:43.188128 30733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:35081 every 8 connection(s)
I20260812 14:47:43.188585 30618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:43.189118 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 30618
I20260812 14:47:43.189445 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:41267
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:35081
--webserver_port=37241
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:47:43.190316 30736 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:47:43.191000 30736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Bootstrap starting.
I20260812 14:47:43.202438 30736 log.cc:824] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:43.204886 30736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:47:43.205262 30736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Bootstrap complete.
I20260812 14:47:43.208547 30736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:43.208894 30736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094f8c03afb84dd3824583df59095c47, State: Initialized, Role: FOLLOWER
I20260812 14:47:43.209049 30736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:43.209112 30736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:43.209158 30736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:43.209203 30736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:47:43.210624 30736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:43.210736 30736 leader_election.cc:302] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 094f8c03afb84dd3824583df59095c47; no voters:
I20260812 14:47:43.210894 30736 leader_election.cc:288] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:47:43.210984 30749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:47:43.211143 30736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:43.211153 30749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [term 2 LEADER]: Becoming Leader. State: Replica: 094f8c03afb84dd3824583df59095c47, State: Running, Role: LEADER
I20260812 14:47:43.211362 30749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } }
I20260812 14:47:43.211656 30749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 094f8c03afb84dd3824583df59095c47. Latest consensus state: current_term: 2 leader_uuid: "094f8c03afb84dd3824583df59095c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } } }
I20260812 14:47:43.211786 30749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:43.211656 30750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "094f8c03afb84dd3824583df59095c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094f8c03afb84dd3824583df59095c47" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35081 } } }
I20260812 14:47:43.211923 30750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:43.211990 30767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:43.212510 30767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:43.212654 30767 catalog_manager.cc:1259] Loaded cluster ID: 22ada8dc746c4a689937926a037e6e30
I20260812 14:47:43.212714 30767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:43.213737 30767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:43.213939 30767 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 094f8c03afb84dd3824583df59095c47: Loaded TSK: 0
I20260812 14:47:43.214157 30767 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:47:43.289728 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:43.289937 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:43.289963 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:43.291141 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:43.291193 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:43.291220 30747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:43.292723 30747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.129.keytab
--principal=oryx/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:41267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=37241
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:43.292980 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:43.293263 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:43.298413 30789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:43.298355 30785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.298629 30747 server_base.cc:1034] running on GCE node
W20260812 14:47:43.298355 30786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.298990 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:43.299288 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:43.300449 30747 hybrid_clock.cc:648] HybridClock initialized: now 1786546063300429 us; error 46 us; skew 500 ppm
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546063, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:43.303790 30747 init.cc:377] Logged in from keytab as oryx/127.13.146.129@KRBTEST.COM (short username oryx)
I20260812 14:47:43.304284 30747 webserver.cc:460] Webserver started at http://127.13.146.129:37241/ using document root <none> and password file <none>
I20260812 14:47:43.304539 30747 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:43.304597 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:43.306042 30747 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:43.306702 30807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.306951 30747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:43.307056 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "b1a68e4ce4c347faae5a37f32e64fa24"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "a939970bbb5414e65745942e628358a7"
server_key_iv: "196794c1219811e0f3de5982096f8071"
server_key_version: "encryptionkey@0"
I20260812 14:47:43.307452 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:43.327898 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:43.330262 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:43.330430 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:43.330688 30747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:43.331001 30747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:43.331054 30747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.331090 30747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:43.331116 30747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.338891 30747 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:41267
I20260812 14:47:43.338924 31023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:41267 every 8 connection(s)
I20260812 14:47:43.339349 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.347609 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 30747
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.347961 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:39049
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:35081
--webserver_port=43445
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:47:43.349068 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.340265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:46361 (local address 127.13.146.190:35081)
0812 14:47:43.340461 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.340468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.340497 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:47:43.341334 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.341340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.341382 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.341491 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.342843 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.343659 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.345055 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.345499 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.345578 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.348934 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:46361: BlockingRecv error: recv got EOF from 127.13.146.129:46361 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260812 14:47:43.349447 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.340131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:46361)
0812 14:47:43.340359 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.340408 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.341079 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.341572 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.341578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.342441 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.342725 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.342733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.343768 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.343769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.344970 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.344973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.345579 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.345580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.345727 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.346240 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.346250 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.349276 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":132,"threads_started":1}
W20260812 14:47:43.349700 31024 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:35081 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.146.190:35081: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.354954 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.349948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:52557)
0812 14:47:43.349984 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.349996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.350042 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.350181 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.350182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.350349 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.350407 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.350409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.351264 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.351265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.352109 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.352112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.352474 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.352475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.352549 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.352976 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.352981 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.354875 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260812 14:47:43.355013 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.350007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:52557 (local address 127.13.146.190:35081)
0812 14:47:43.350050 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.350052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.350064 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:47:43.350088 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.350089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.350100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.350156 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.350487 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.351174 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.352200 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.352381 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.352474 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.354951 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:52557: BlockingRecv error: recv got EOF from 127.13.146.129:52557 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.359371 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.355251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:49281)
0812 14:47:43.355293 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.355307 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.355356 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.355497 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.355498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.355622 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.355647 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.355649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.356256 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.356257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.356733 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.356734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.357063 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.357064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.357117 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.357518 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.357524 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.359297 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:47:43.359424 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.355294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:49281 (local address 127.13.146.190:35081)
0812 14:47:43.355347 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.355349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.355359 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:43.355414 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.355415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.355426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.355497 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.355725 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.356161 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.356810 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.356993 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.357064 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.359363 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:49281: BlockingRecv error: recv got EOF from 127.13.146.129:49281 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 14:47:43.359498 31024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:47:43.433825 31032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:43.434191 31032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:43.434274 31032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:43.435653 31032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:43.435704 31032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:43.435732 31032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:43.437301 31032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.130.keytab
--principal=oryx/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=43445
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:43.437556 31032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:43.437803 31032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:43.443159 31039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:43.443277 31040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:43.443158 31042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.443331 31032 server_base.cc:1034] running on GCE node
I20260812 14:47:43.443529 31032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:43.443729 31032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:43.444883 31032 hybrid_clock.cc:648] HybridClock initialized: now 1786546063444870 us; error 27 us; skew 500 ppm
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546063, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:43.447530 31032 init.cc:377] Logged in from keytab as oryx/127.13.146.130@KRBTEST.COM (short username oryx)
I20260812 14:47:43.447990 31032 webserver.cc:460] Webserver started at http://127.13.146.130:43445/ using document root <none> and password file <none>
I20260812 14:47:43.448180 31032 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:43.448246 31032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:43.449725 31032 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:43.450271 31053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.450452 31032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:43.450552 31032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "67acd14ef44c44ae9a5d7ee1918e2877"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "ea05dedae6d69ccaa7fb3c9c3e664e86"
server_key_iv: "b46364e3965dde7ac246916400db9991"
server_key_version: "encryptionkey@0"
I20260812 14:47:43.450886 31032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:43.483417 31032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:43.485483 31032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:43.485646 31032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:43.485891 31032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:43.486224 31032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:43.486259 31032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.486282 31032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:43.486330 31032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.492046 31032 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:39049
I20260812 14:47:43.492100 31243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:39049 every 8 connection(s)
I20260812 14:47:43.492440 31032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:43.495038 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 31032
I20260812 14:47:43.495321 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:41907
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:35081
--webserver_port=46407
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.500178 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.493159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:33971)
0812 14:47:43.493360 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.493398 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.493927 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.494184 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.494187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.494803 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.494994 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.494998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.495813 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.495814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.496605 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.496607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.496986 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.496987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.497120 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.497659 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.497667 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.500033 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
W20260812 14:47:43.500242 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.493164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:33971 (local address 127.13.146.190:35081)
0812 14:47:43.493252 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.493255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.493483 (+ 228us) server_negotiation.cc:407] Connection header received
0812 14:47:43.494032 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.494034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.494052 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.494111 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.495099 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.495713 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.496697 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.496890 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.496974 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.500156 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:33971: BlockingRecv error: recv got EOF from 127.13.146.130:33971 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 14:47:43.500538 31244 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:35081 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.146.190:35081: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.504956 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.500800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:57433)
0812 14:47:43.500845 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.500855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.500910 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.501052 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.501052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.501189 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.501282 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.501284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.501882 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.501883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.502385 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.502387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.502736 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.502737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.502803 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.503214 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.503219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.504874 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:47:43.505013 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.500855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57433 (local address 127.13.146.190:35081)
0812 14:47:43.500914 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.500916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.500925 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:43.500971 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.500972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.500982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.501049 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.501354 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.501807 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.502480 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.502666 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.502736 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.504949 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:57433: BlockingRecv error: recv got EOF from 127.13.146.130:57433 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.509279 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.505355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:42163)
0812 14:47:43.505399 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.505410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.505458 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.505576 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.505576 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.505704 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.505733 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.505734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.506330 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.506331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.506817 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.506818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.507168 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.507169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.507218 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.507611 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.507616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.509172 (+ 1556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:47:43.509341 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.505433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42163 (local address 127.13.146.190:35081)
0812 14:47:43.505485 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.505487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.505496 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:43.505517 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.505518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.505526 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.505574 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.505810 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.506256 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.506910 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.507101 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.507170 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.509283 (+ 2113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:42163: BlockingRecv error: recv got EOF from 127.13.146.130:42163 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 14:47:43.509527 31244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:47:43.581995 31247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:43.582211 31247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:43.582235 31247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:43.583410 31247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:43.583468 31247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:43.583493 31247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:43.584942 31247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/krb5kdc/oryx_127.13.146.131.keytab
--principal=oryx/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:41907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=46407
--tserver_master_addrs=127.13.146.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:43.585191 31247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:43.585448 31247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:43.590587 31262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.590659 31247 server_base.cc:1034] running on GCE node
W20260812 14:47:43.590654 31258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:43.590651 31255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:43.591081 31247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:43.591342 31247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:43.592499 31247 hybrid_clock.cc:648] HybridClock initialized: now 1786546063592482 us; error 39 us; skew 500 ppm
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546063, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:47:43.595099 31247 init.cc:377] Logged in from keytab as oryx/127.13.146.131@KRBTEST.COM (short username oryx)
I20260812 14:47:43.595527 31247 webserver.cc:460] Webserver started at http://127.13.146.131:46407/ using document root <none> and password file <none>
I20260812 14:47:43.595757 31247 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:43.595837 31247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:43.597260 31247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:47:43.597885 31275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.598061 31247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:43.598158 31247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "a98a0287498e4a90af30bd62656947e9"
format_stamp: "Formatted at 2026-08-12 14:47:42 on dist-test-slave-8hhm"
server_key: "b4d68fef0a1e94f048eb13663d6416c5"
server_key_iv: "52824459709615f486960f8159341fe1"
server_key_version: "encryptionkey@0"
I20260812 14:47:43.598546 31247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:43.633922 31247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:43.636080 31247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:43.636271 31247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:43.636538 31247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:43.636870 31247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:43.636924 31247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.636966 31247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:43.637006 31247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:43.643041 31247 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41907
I20260812 14:47:43.643113 31487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41907 every 8 connection(s)
I20260812 14:47:43.643481 31247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:47:43.651117 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.644208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:38695 (local address 127.13.146.190:35081)
0812 14:47:43.644280 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.644282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.644489 (+ 207us) server_negotiation.cc:407] Connection header received
0812 14:47:43.645023 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.645024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.645034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.645093 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.646121 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.646605 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.647610 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.647790 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.647840 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.651056 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:38695: BlockingRecv error: recv got EOF from 127.13.146.131:38695 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:47:43.651193 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.644193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:38695)
0812 14:47:43.644370 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.644411 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.644939 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.645163 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.645167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.645830 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.646030 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.646036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.646683 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.646684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.647534 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.647536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.647860 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.647860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.647963 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.648528 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.648538 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.650978 (+ 2440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
W20260812 14:47:43.651435 31488 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:35081 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.146.190:35081: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
I20260812 14:47:43.652043 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 31247
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.656615 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.651673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:46839)
0812 14:47:43.651729 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.651750 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.651791 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.652002 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.652003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.652172 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.652234 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.652236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.653002 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.653003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.653703 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.653705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.654077 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.654078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.654148 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.654587 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.654592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.656530 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 14:47:43.656685 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.651844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:46839 (local address 127.13.146.190:35081)
0812 14:47:43.651881 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.651883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.651890 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:47:43.651913 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.651914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.651929 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.651977 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.652312 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.652908 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.653807 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.653997 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.654067 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.656602 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:46839: BlockingRecv error: recv got EOF from 127.13.146.131:46839 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.660951 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.652617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.13.146.190:35081)
0812 14:47:43.652736 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.652738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.652886 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:47:43.652976 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.652977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.652989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.653109 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.653761 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.654246 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.655352 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.655708 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.655794 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.657265 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:43.657271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:43.657271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:47:43.657286 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:43.659245 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:43.659690 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:43.659695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:43.659695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:43.659848 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:47:43.660056 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:47:43.660057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:47:43.660058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:47:43.660380 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:43.660503 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:43.660669 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 14:47:43.660815 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
Aug 12 14:47:43 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:43.662025 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:43.656955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:35135)
0812 14:47:43.656997 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:43.657006 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:43.657052 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:43.657232 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:43.657233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:43.657381 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:43.657419 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.657421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.658250 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.658251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.658999 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:43.659002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.659381 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.659382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:43.659422 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.659767 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:43.659772 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:43.661946 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:47:43.662062 31030 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:43.656994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:35135 (local address 127.13.146.190:35081)
0812 14:47:43.657070 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:43.657079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:43.657089 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:43.657112 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:43.657113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:43.657124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:43.657191 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:43.657502 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.658162 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.659102 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:43.659297 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:43.659372 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:43.662007 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:35135: BlockingRecv error: recv got EOF from 127.13.146.131:35135 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 14:47:43.662189 31488 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:44.365084 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:44.360072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:40039)
0812 14:47:44.360253 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:44.360265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:44.360330 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:44.360478 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:44.360479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:44.360688 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:44.360741 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.360743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.361682 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.361683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.362312 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.362314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.362677 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.362678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.362718 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.363213 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:44.363219 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:44.365013 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
W20260812 14:47:44.365161 31499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:44.360198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:40039 (local address 127.13.146.190:35081)
0812 14:47:44.360335 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:44.360337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:44.360350 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:44.360390 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:44.360391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:44.360401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:44.360461 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:44.360810 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.361562 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.362381 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.362614 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.362668 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.365102 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:40039: BlockingRecv error: recv got EOF from 127.13.146.129:40039 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:44.514983 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:44.509990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:43323)
0812 14:47:44.510163 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:44.510177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:44.510277 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:44.510472 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:44.510474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:44.510678 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:44.510779 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.510782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.511523 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.511524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.512087 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.512088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.512477 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.512478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.512533 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.513035 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:44.513046 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:44.514866 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
W20260812 14:47:44.515020 31499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:44.510095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:43323 (local address 127.13.146.190:35081)
0812 14:47:44.510178 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:44.510180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:44.510253 (+ 73us) server_negotiation.cc:407] Connection header received
0812 14:47:44.510355 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:44.510356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:44.510371 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:44.510452 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:44.510859 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.511435 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.512171 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.512406 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.512479 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.514955 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:43323: BlockingRecv error: recv got EOF from 127.13.146.130:43323 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:44 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:44.667757 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:44.662624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:51803)
0812 14:47:44.662828 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:44.662842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:44.662935 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:44.663112 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:44.663113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:44.663308 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:44.663357 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.663359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.664062 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.664063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.664644 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:44.664646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.665004 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.665005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:44.665062 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.665684 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:44.665690 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:44.667646 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
W20260812 14:47:44.667793 31499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:44.662732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51803 (local address 127.13.146.190:35081)
0812 14:47:44.662817 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:44.662820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:44.662915 (+ 95us) server_negotiation.cc:407] Connection header received
0812 14:47:44.662997 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:44.662998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:44.663013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:44.663081 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:44.663460 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.663977 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.664734 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:44.664931 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:44.665005 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:44.667726 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:51803: BlockingRecv error: recv got EOF from 127.13.146.131:51803 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:45.370536 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:45.365695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:54627)
0812 14:47:45.365844 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:45.365857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:45.365925 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:45.366070 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:45.366071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:45.366263 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:45.366313 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.366316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.367067 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.367069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.367599 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.367600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.368005 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.368006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.368073 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.368583 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:45.368588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:45.370435 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260812 14:47:45.370594 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:45.365728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:54627 (local address 127.13.146.190:35081)
0812 14:47:45.365870 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:45.365873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:45.365903 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:47:45.365985 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:45.365986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:45.365999 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:45.366058 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:45.366396 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.366986 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.367682 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.367932 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.368004 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.370519 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:54627: BlockingRecv error: recv got EOF from 127.13.146.129:54627 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:45.520316 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:45.515500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:59781)
0812 14:47:45.515660 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:45.515671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:45.515739 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:45.515886 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:45.515887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:45.516060 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:45.516110 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.516112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.516749 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.516750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.517294 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.517295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.517715 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.517717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.517784 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.518381 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:45.518391 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:45.520215 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260812 14:47:45.520368 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:45.515604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:59781 (local address 127.13.146.190:35081)
0812 14:47:45.515674 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:45.515676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:45.515739 (+ 63us) server_negotiation.cc:407] Connection header received
0812 14:47:45.515809 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:45.515810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:45.515822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:45.515869 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:45.516174 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.516673 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.517361 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.517630 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.517711 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.520304 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:59781: BlockingRecv error: recv got EOF from 127.13.146.130:59781 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:45 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:45.673036 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:45.668260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:58135)
0812 14:47:45.668416 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:45.668428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:45.668484 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:45.668630 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:45.668631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:45.668816 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:45.668867 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.668869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.669570 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.669571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.670110 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:45.670112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.670495 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.670495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:45.670534 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.671124 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:45.671129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:45.672974 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
W20260812 14:47:45.673169 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:45.668383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:58135 (local address 127.13.146.190:35081)
0812 14:47:45.668433 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:45.668434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:45.668470 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:47:45.668537 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:45.668537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:45.668549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:45.668612 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:45.668933 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.669494 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.670223 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:45.670421 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:45.670487 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:45.673086 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:58135: BlockingRecv error: recv got EOF from 127.13.146.131:58135 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:46.375922 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:46.371188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:54703)
0812 14:47:46.371364 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:46.371374 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:46.371427 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:46.371574 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:46.371575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:46.371781 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:46.371833 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.371835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.372560 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.372561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.373093 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.373095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.373576 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.373577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.373623 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.374080 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:46.374084 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:46.375856 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260812 14:47:46.376055 31511 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:46.371272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:54703 (local address 127.13.146.190:35081)
0812 14:47:46.371418 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:46.371420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:46.371430 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:46.371475 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:46.371475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:46.371486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:46.371558 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:46.371934 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.372480 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.373208 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.373494 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.373555 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.375948 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:54703: BlockingRecv error: recv got EOF from 127.13.146.129:54703 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:46.525120 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:46.520797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:59203)
0812 14:47:46.520942 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:46.520953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:46.521011 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:46.521161 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:46.521161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:46.521359 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:46.521409 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.521410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.522049 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.522050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.522577 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.522578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.522893 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.522893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.522934 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.523380 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:46.523385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:46.525059 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260812 14:47:46.525249 31511 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:46.520909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:59203 (local address 127.13.146.190:35081)
0812 14:47:46.520958 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:46.520959 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:46.520999 (+ 40us) server_negotiation.cc:407] Connection header received
0812 14:47:46.521072 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:46.521072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:46.521083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:46.521142 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:46.521479 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.521972 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.522647 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.522828 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.522880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.525154 (+ 2274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:59203: BlockingRecv error: recv got EOF from 127.13.146.130:59203 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:46 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:46.677942 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:46.673503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:55981)
0812 14:47:46.673627 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:46.673638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:46.673703 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:46.673844 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:46.673845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:46.674019 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:46.674066 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.674068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.674701 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.674702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.675222 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:46.675223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.675546 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.675547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:46.675586 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.676027 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:46.676032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:46.677879 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260812 14:47:46.678071 31511 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:46.673555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:55981 (local address 127.13.146.190:35081)
0812 14:47:46.673602 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:46.673604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:46.673686 (+ 82us) server_negotiation.cc:407] Connection header received
0812 14:47:46.673772 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:46.673773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:46.673785 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:46.673827 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:46.674130 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.674627 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.675291 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:46.675483 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:46.675537 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:46.677973 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:55981: BlockingRecv error: recv got EOF from 127.13.146.131:55981 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:47.380877 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:47.376448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:59905)
0812 14:47:47.376598 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:47.376613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:47.376680 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:47.376848 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:47.376849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:47.377024 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:47.377079 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.377081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.377774 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.377775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.378331 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.378333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.378661 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.378662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.378707 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.379155 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:47.379160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:47.380818 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":60,"threads_started":1}
W20260812 14:47:47.380995 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:47.376544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:59905 (local address 127.13.146.190:35081)
0812 14:47:47.376722 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:47.376724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:47.376733 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:47.376756 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:47.376757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:47.376768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:47.376826 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:47.377143 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.377693 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.378412 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.378604 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.378649 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.380914 (+ 2265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:59905: BlockingRecv error: recv got EOF from 127.13.146.129:59905 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:47.530603 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:47.525724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:55077)
0812 14:47:47.525933 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:47.525944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:47.526012 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:47.526150 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:47.526150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:47.526329 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:47.526379 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.526381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.527041 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.527042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.527582 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.527584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.527969 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.527969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.528015 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.528622 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:47.528627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:47.530542 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":125,"threads_started":1}
W20260812 14:47:47.530769 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:47.525817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:55077 (local address 127.13.146.190:35081)
0812 14:47:47.525897 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:47.525899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:47.526026 (+ 127us) server_negotiation.cc:407] Connection header received
0812 14:47:47.526072 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:47.526073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:47.526086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:47.526145 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:47.526445 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.526946 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.527689 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.527888 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.527951 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.530689 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:55077: BlockingRecv error: recv got EOF from 127.13.146.130:55077 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:47 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:47.682962 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:47.678445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:44205)
0812 14:47:47.678607 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:47.678619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:47.678692 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:47.678826 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:47.678827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:47.678998 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:47.679074 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.679076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.679693 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.679694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.680244 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:47.680246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.680562 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.680563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:47.680602 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.681096 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:47.681101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:47.682898 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
W20260812 14:47:47.683033 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:47.678562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:44205 (local address 127.13.146.190:35081)
0812 14:47:47.678612 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:47.678614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:47.678674 (+ 60us) server_negotiation.cc:407] Connection header received
0812 14:47:47.678740 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:47.678741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:47.678752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:47.678802 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:47.679142 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.679623 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.680329 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:47.680500 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:47.680548 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:47.682985 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:44205: BlockingRecv error: recv got EOF from 127.13.146.131:44205 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:48.390234 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:48.381628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:34541)
0812 14:47:48.381848 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:48.381862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:48.381930 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:48.382088 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:48.382089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:48.382271 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:48.382322 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.382324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.382979 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.382980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.383520 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.383521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.383879 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.383879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.383918 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.388192 (+ 4274us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:48.388197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:48.390166 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":122,"threads_started":1}
W20260812 14:47:48.390377 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:48.381736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:34541 (local address 127.13.146.190:35081)
0812 14:47:48.381917 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:48.381919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:48.381929 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:48.381977 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:48.381978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:48.381990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:48.382074 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:48.382387 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.382914 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.383596 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.383808 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.383868 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.390279 (+ 6411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:34541: BlockingRecv error: recv got EOF from 127.13.146.129:34541 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:48.536337 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:48.531256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:51519)
0812 14:47:48.531457 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:48.531476 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:48.531547 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:48.531687 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:48.531688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:48.531875 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:48.531928 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.531929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.532587 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.532588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.533103 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.533105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.533494 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.533496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.533561 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.534068 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:48.534074 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:48.536277 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":107,"threads_started":1}
W20260812 14:47:48.536501 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:48.531417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:51519 (local address 127.13.146.190:35081)
0812 14:47:48.531466 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:48.531468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:48.531531 (+ 63us) server_negotiation.cc:407] Connection header received
0812 14:47:48.531610 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:48.531611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:48.531621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:48.531663 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:48.531995 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.532514 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.533186 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.533408 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.533467 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.536403 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:51519: BlockingRecv error: recv got EOF from 127.13.146.130:51519 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:48 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:48.688445 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:48.683625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:39377)
0812 14:47:48.683838 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:48.683854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:48.683967 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:48.684147 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:48.684148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:48.684340 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:48.684388 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.684390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.685055 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.685056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.685674 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:48.685676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.686040 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.686042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:48.686104 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.686559 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:48.686564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:48.688348 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
W20260812 14:47:48.688503 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:48.683779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:39377 (local address 127.13.146.190:35081)
0812 14:47:48.683869 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:48.683871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:48.683953 (+ 82us) server_negotiation.cc:407] Connection header received
0812 14:47:48.684027 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:48.684028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:48.684042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:48.684111 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:48.684473 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.684968 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.685762 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:48.685952 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:48.686039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:48.688438 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:39377: BlockingRecv error: recv got EOF from 127.13.146.131:39377 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:49.395782 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:49.390905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:43949)
0812 14:47:49.391075 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:49.391090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:49.391177 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:49.391346 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:49.391347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:49.391607 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:49.391667 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.391669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.392361 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.392361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.392889 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.392890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.393276 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.393277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.393338 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.393819 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:49.393824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:49.395721 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
W20260812 14:47:49.395920 31535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:49.390995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:43949 (local address 127.13.146.190:35081)
0812 14:47:49.391141 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:49.391144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:49.391163 (+ 19us) server_negotiation.cc:407] Connection header received
0812 14:47:49.391247 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:49.391247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:49.391261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:49.391335 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:49.391762 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.392289 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.392959 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.393167 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.393276 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.395831 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:43949: BlockingRecv error: recv got EOF from 127.13.146.129:43949 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:49.541498 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:49.536795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:42623)
0812 14:47:49.536946 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:49.536957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:49.537013 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:49.537179 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:49.537180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:49.537400 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:49.537451 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.537453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.538060 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.538061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.538572 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.538574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.538881 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.538882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.538920 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.539463 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:49.539468 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:49.541398 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
W20260812 14:47:49.541551 31535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:49.536876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42623 (local address 127.13.146.190:35081)
0812 14:47:49.536943 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:49.536944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:49.536999 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:47:49.537094 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:49.537095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:49.537106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:49.537162 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:49.537517 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.537989 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.538641 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.538819 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.538868 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.541487 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:42623: BlockingRecv error: recv got EOF from 127.13.146.130:42623 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:49 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:49.693743 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:49.688952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:59887)
0812 14:47:49.689134 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:49.689148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:49.689266 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:49.689469 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:49.689470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:49.689687 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:49.689747 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.689749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.690416 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.690416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.690934 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:49.690936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.691291 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.691292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:49.691331 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.691795 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:49.691800 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:49.693680 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":120,"threads_started":1}
W20260812 14:47:49.693871 31535 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:49.689063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:59887 (local address 127.13.146.190:35081)
0812 14:47:49.689139 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:49.689141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:49.689242 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:47:49.689366 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:49.689367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:49.689383 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:49.689459 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:49.689840 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.690337 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.691026 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:49.691218 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:49.691277 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:49.693768 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:59887: BlockingRecv error: recv got EOF from 127.13.146.131:59887 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:50.400924 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:50.396281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:60589)
0812 14:47:50.396439 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:50.396449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:50.396502 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:50.396650 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:50.396650 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:50.396823 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:50.396875 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.396877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.397536 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.397538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.398142 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.398144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.398465 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.398466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.398509 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.399039 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:50.399044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:50.400865 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260812 14:47:50.401116 31539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:50.396359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:60589 (local address 127.13.146.190:35081)
0812 14:47:50.396495 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:50.396497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:50.396505 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:47:50.396561 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:50.396561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:50.396571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:50.396621 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:50.396945 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.397464 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.398215 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.398402 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.398452 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.401036 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:60589: BlockingRecv error: recv got EOF from 127.13.146.129:60589 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:50.546902 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:50.542007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:53993)
0812 14:47:50.542194 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:50.542209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:50.542297 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:50.542440 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:50.542441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:50.542632 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:50.542689 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.542691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.543435 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.543436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.544001 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.544003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.544347 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.544348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.544400 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.544855 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:50.544860 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:50.546813 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
W20260812 14:47:50.546984 31539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:50.542164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:53993 (local address 127.13.146.190:35081)
0812 14:47:50.542237 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:50.542239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:50.542274 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:47:50.542347 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:50.542348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:50.542360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:50.542410 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:50.542759 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.543347 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.544082 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.544263 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.544314 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.546917 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:53993: BlockingRecv error: recv got EOF from 127.13.146.130:53993 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:50 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:50.698990 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:50.694254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:40981)
0812 14:47:50.694461 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:50.694474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:50.694553 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:50.694694 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:50.694695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:50.694870 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:50.694921 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.694923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.695556 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.695557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.696092 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:50.696093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.696419 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.696420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:50.696459 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.696930 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:50.696934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:50.698920 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
W20260812 14:47:50.699172 31539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:50.694350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:40981 (local address 127.13.146.190:35081)
0812 14:47:50.694463 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:50.694465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:50.694550 (+ 85us) server_negotiation.cc:407] Connection header received
0812 14:47:50.694602 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:50.694603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:50.694617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:50.694668 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:50.694995 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.695485 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.696173 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:50.696354 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:50.696403 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:50.699046 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:40981: BlockingRecv error: recv got EOF from 127.13.146.131:40981 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:51.406306 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:51.401557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:44549)
0812 14:47:51.401703 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:51.401714 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:51.401775 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:51.402092 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:51.402094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:51.402288 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:51.402348 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.402349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.402984 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.402984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.403540 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.403542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.403868 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.403868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.403908 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.404368 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:51.404372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:51.406235 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260812 14:47:51.406391 31546 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:51.401699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:44549 (local address 127.13.146.190:35081)
0812 14:47:51.401910 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:51.401912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:51.401921 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:51.401948 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:51.401949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:51.401962 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:51.402029 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:51.402413 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.402913 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.403608 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.403803 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.403852 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.406337 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:44549: BlockingRecv error: recv got EOF from 127.13.146.129:44549 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:51.552304 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:51.547672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:59171)
0812 14:47:51.547823 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:51.547835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:51.547902 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:51.548034 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:51.548034 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:51.548219 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:51.548271 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.548273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.549024 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.549025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.549580 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.549581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.549927 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.549928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.549967 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.550411 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:51.550415 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:51.552233 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260812 14:47:51.552445 31546 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:51.547771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:59171 (local address 127.13.146.190:35081)
0812 14:47:51.547832 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:51.547834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:51.547887 (+ 53us) server_negotiation.cc:407] Connection header received
0812 14:47:51.547961 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:51.547962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:51.547973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:51.548015 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:51.548335 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.548953 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.549656 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.549845 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.549901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.552359 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:59171: BlockingRecv error: recv got EOF from 127.13.146.130:59171 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:51 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:51.704303 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:51.699536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:54993)
0812 14:47:51.699690 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:51.699701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:51.699775 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:51.699895 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:51.699896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:51.700065 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:51.700115 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.700117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.700764 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.700765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.701380 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:51.701382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.701726 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.701727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:51.701767 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.702338 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:51.702342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:51.704241 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260812 14:47:51.704488 31546 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:51.699595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:54993 (local address 127.13.146.190:35081)
0812 14:47:51.699644 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:51.699647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:51.699775 (+ 128us) server_negotiation.cc:407] Connection header received
0812 14:47:51.699821 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:51.699821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:51.699834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:51.699887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:51.700173 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.700672 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.701454 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:51.701646 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:51.701702 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:51.704424 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:54993: BlockingRecv error: recv got EOF from 127.13.146.131:54993 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:52.411932 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:52.406875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:59513)
0812 14:47:52.407070 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:52.407082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:52.407173 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:52.407345 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:52.407346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:52.407528 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:52.407581 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.407583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.408239 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.408239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.408762 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.408763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.409081 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.409081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.409121 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.409640 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:52.409645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:52.411826 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
W20260812 14:47:52.412007 31552 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:52.406945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:59513 (local address 127.13.146.190:35081)
0812 14:47:52.407046 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:52.407049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:52.407169 (+ 120us) server_negotiation.cc:407] Connection header received
0812 14:47:52.407245 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:52.407246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:52.407256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:52.407328 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:52.407640 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.408160 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.408831 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.409018 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.409065 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.411925 (+ 2860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:59513: BlockingRecv error: recv got EOF from 127.13.146.129:59513 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:52.557695 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:52.552774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:46045)
0812 14:47:52.552929 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:52.552941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:52.552999 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:52.553173 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:52.553174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:52.553365 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:52.553447 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.553449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.554156 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.554157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.554750 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.554753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.555129 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.555130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.555198 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.555649 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:52.555653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:52.557596 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260812 14:47:52.557739 31552 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:52.552908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:46045 (local address 127.13.146.190:35081)
0812 14:47:52.552980 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:52.552984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:52.552998 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:47:52.553049 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:52.553049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:52.553095 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:52.553152 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:52.553540 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.554067 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.554837 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.555039 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.555126 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.557678 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:46045: BlockingRecv error: recv got EOF from 127.13.146.130:46045 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:52 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:52.709831 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:52.704752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:41199)
0812 14:47:52.704899 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:52.704911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:52.704987 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:52.705123 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:52.705124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:52.705388 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:52.705455 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.705458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.706185 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.706186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.706705 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:52.706706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.707110 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.707111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:52.707154 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.707677 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:52.707681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:52.709715 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260812 14:47:52.710000 31552 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:52.704861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:41199 (local address 127.13.146.190:35081)
0812 14:47:52.704910 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:52.704912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:52.704987 (+ 75us) server_negotiation.cc:407] Connection header received
0812 14:47:52.705041 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:52.705041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:52.705054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:52.705095 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:52.705581 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.706104 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.706789 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:52.706979 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:52.707075 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:52.709911 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:41199: BlockingRecv error: recv got EOF from 127.13.146.131:41199 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:53.422158 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:53.412362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:46117)
0812 14:47:53.412519 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:53.412531 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:53.412595 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:53.412779 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:53.412780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:53.412957 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:53.413010 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.413012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.413726 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.413727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.414291 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.414292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.414619 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.414620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.414659 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.420026 (+ 5367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:53.420032 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:53.422082 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260812 14:47:53.422302 31556 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:53.412455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:46117 (local address 127.13.146.190:35081)
0812 14:47:53.412560 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:53.412562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:53.412572 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:53.412644 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:53.412644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:53.412656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:53.412718 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:53.413085 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.413649 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.414364 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.414555 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.414605 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.422196 (+ 7591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:46117: BlockingRecv error: recv got EOF from 127.13.146.129:46117 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":54,"threads_started":1}
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:53.562937 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:53.558216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:60743)
0812 14:47:53.558366 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:53.558376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:53.558435 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:53.558581 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:53.558581 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:53.558757 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:53.558808 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.558810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.559477 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.559478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.559994 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.559995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.560380 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.560381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.560423 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.560897 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:53.560901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:53.562839 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
W20260812 14:47:53.562992 31556 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:53.558314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:60743 (local address 127.13.146.190:35081)
0812 14:47:53.558374 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:53.558376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:53.558423 (+ 47us) server_negotiation.cc:407] Connection header received
0812 14:47:53.558501 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:53.558502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:53.558514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:53.558566 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:53.558877 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.559405 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.560102 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.560306 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.560371 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.562930 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:60743: BlockingRecv error: recv got EOF from 127.13.146.130:60743 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:53 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:53.715286 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:53.710366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:39211)
0812 14:47:53.710514 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:53.710525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:53.710592 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:53.710751 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:53.710751 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:53.710961 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:53.711011 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.711012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.711810 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.711811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.712356 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:53.712357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.712723 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.712758 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:53.712822 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.713376 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:53.713380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:53.715226 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
W20260812 14:47:53.715390 31556 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:53.710477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:39211 (local address 127.13.146.190:35081)
0812 14:47:53.710549 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:53.710552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:53.710591 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:47:53.710662 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:53.710663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:53.710674 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:53.710734 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:53.711115 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.711716 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.712449 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:53.712647 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:53.712710 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:53.715319 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:39211: BlockingRecv error: recv got EOF from 127.13.146.131:39211 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:54.427382 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:54.422710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:38395)
0812 14:47:54.422875 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:54.422885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:54.422936 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:54.423114 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:54.423115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:54.423303 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:54.423353 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.423355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.423989 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.423989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.424521 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.424522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.424873 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.424874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.424913 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.425482 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:54.425487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:54.427313 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
W20260812 14:47:54.427513 31561 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:54.422813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:38395 (local address 127.13.146.190:35081)
0812 14:47:54.422947 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:54.422949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:54.422962 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:47:54.422986 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:54.422986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:54.422998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:54.423103 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:54.423423 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.423920 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.424602 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.424804 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.424861 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.427405 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:38395: BlockingRecv error: recv got EOF from 127.13.146.129:38395 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:54.568212 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:54.563483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:60789)
0812 14:47:54.563656 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:54.563667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:54.563727 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:54.563848 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:54.563849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:54.564031 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:54.564079 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.564080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.564721 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.564722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.565347 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.565350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.565659 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.565660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.565699 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.566253 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:54.566257 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:54.568134 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":129,"threads_started":1}
W20260812 14:47:54.568272 31561 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:54.563545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:60789 (local address 127.13.146.190:35081)
0812 14:47:54.563612 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:54.563615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:54.563727 (+ 112us) server_negotiation.cc:407] Connection header received
0812 14:47:54.563788 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:54.563789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:54.563801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:54.563842 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:54.564145 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.564653 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.565418 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.565598 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.565648 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.568224 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:60789: BlockingRecv error: recv got EOF from 127.13.146.130:60789 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:54 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:54.720965 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:54.715868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:54401)
0812 14:47:54.716046 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:54.716058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:54.716131 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:54.716330 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:54.716332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:54.716575 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:54.716647 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.716650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.717478 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.717479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.718056 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:54.718058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.718429 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.718430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:54.718490 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.719020 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:54.719026 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:54.720864 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
W20260812 14:47:54.721100 31561 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:54.715926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:54401 (local address 127.13.146.190:35081)
0812 14:47:54.716000 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:54.716003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:54.716196 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:47:54.716232 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:54.716234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:54.716252 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:54.716319 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:54.716722 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.717393 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.718147 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:54.718353 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:54.718431 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:54.721016 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:54401: BlockingRecv error: recv got EOF from 127.13.146.131:54401 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:55.433383 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:55.427880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:53393)
0812 14:47:55.428118 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:55.428131 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:55.428211 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:55.428611 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:55.428613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:55.428823 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:55.428874 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.428877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.429773 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.429775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.430323 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.430325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.430707 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.430709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.430777 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.431274 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:55.431279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:55.433288 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
W20260812 14:47:55.433429 31569 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:55.428025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:53393 (local address 127.13.146.190:35081)
0812 14:47:55.428180 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:55.428182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:55.428192 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:55.428275 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:55.428276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:55.428288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:55.428341 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:55.428981 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.429682 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.430416 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.430628 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.430706 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.433358 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:53393: BlockingRecv error: recv got EOF from 127.13.146.129:53393 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:55.573578 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:55.568742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:54907)
0812 14:47:55.568917 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:55.568929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:55.568994 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:55.569128 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:55.569128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:55.569378 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:55.569428 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.569430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.570086 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.570087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.570614 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.570615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.571022 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.571023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.571063 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.571602 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:55.571606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:55.573482 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
W20260812 14:47:55.573715 31569 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:55.568791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:54907 (local address 127.13.146.190:35081)
0812 14:47:55.568866 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:55.568869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:55.568995 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:47:55.569055 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:55.569056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:55.569068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:55.569114 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:55.569492 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.570014 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.570731 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.570948 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.571012 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.573651 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:54907: BlockingRecv error: recv got EOF from 127.13.146.130:54907 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:55 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:55.726373 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:55.721489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:47051)
0812 14:47:55.721679 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:55.721695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:55.721783 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:55.721951 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:55.721951 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:55.722169 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:55.722216 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.722218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.722906 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.722907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.723458 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:55.723460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.723832 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.723833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:55.723892 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.724373 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:55.724378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:55.726301 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
W20260812 14:47:55.726498 31569 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:55.721651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47051 (local address 127.13.146.190:35081)
0812 14:47:55.721718 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:55.721720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:55.721758 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:47:55.721864 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:55.721865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:55.721881 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:55.721930 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:55.722308 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.722815 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.723550 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:55.723753 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:55.723833 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:55.726414 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:47051: BlockingRecv error: recv got EOF from 127.13.146.131:47051 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:56.438831 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:56.433891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:45237)
0812 14:47:56.434050 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:56.434061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:56.434115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:56.434261 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:56.434262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:56.434434 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:56.434482 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.434483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.435228 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.435229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.435761 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.435763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.436118 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.436119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.436158 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.436670 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:56.436675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:56.438717 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
W20260812 14:47:56.438887 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:56.433983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:45237 (local address 127.13.146.190:35081)
0812 14:47:56.434098 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:56.434100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:56.434110 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:47:56.434170 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:56.434170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:56.434181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:56.434252 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:56.434631 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.435145 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.435842 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.436039 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.436095 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.438803 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:45237: BlockingRecv error: recv got EOF from 127.13.146.129:45237 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:56.578966 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:56.574122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:42337)
0812 14:47:56.574280 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:56.574298 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:56.574360 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:56.574502 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:56.574503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:56.574680 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:56.574730 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.574732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.575365 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.575366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.575918 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.575919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.576241 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.576242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.576286 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.576857 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:56.576862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:56.578850 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260812 14:47:56.578996 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:56.574238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42337 (local address 127.13.146.190:35081)
0812 14:47:56.574306 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:56.574308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:56.574344 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:47:56.574415 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:56.574416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:56.574427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:56.574483 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:56.574798 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.575294 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.575987 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.576166 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.576218 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.578931 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:42337: BlockingRecv error: recv got EOF from 127.13.146.130:42337 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:56 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:56.731593 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:56.726886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:40427)
0812 14:47:56.727043 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:56.727055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:56.727130 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:56.727279 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:56.727279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:56.727463 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:56.727520 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.727521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.728173 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.728175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.728730 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:56.728732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.729054 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.729055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:56.729098 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.729575 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:56.729580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:56.731523 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260812 14:47:56.731665 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:56.726958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:40427 (local address 127.13.146.190:35081)
0812 14:47:56.727025 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:56.727027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:56.727121 (+ 94us) server_negotiation.cc:407] Connection header received
0812 14:47:56.727194 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:56.727194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:56.727207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:56.727251 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:56.727579 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.728069 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.728807 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:56.728985 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:56.729036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:56.731614 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:40427: BlockingRecv error: recv got EOF from 127.13.146.131:40427 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:57.444031 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:57.439355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:43701)
0812 14:47:57.439516 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:57.439527 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:57.439594 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:57.439726 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:57.439727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:57.439910 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:57.439962 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.439964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.440606 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.440607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.441171 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.441173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.441586 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.441587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.441633 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.442126 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:57.442131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:57.443972 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
W20260812 14:47:57.444139 31582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:57.439456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:43701 (local address 127.13.146.190:35081)
0812 14:47:57.439598 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:57.439600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:57.439609 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:47:57.439636 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:57.439636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:57.439647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:57.439724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:57.440033 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.440527 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.441324 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.441515 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.441567 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.444083 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:43701: BlockingRecv error: recv got EOF from 127.13.146.129:43701 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:57.584579 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:57.579541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:34171)
0812 14:47:57.579730 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:57.579741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:57.579837 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:57.580015 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:57.580016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:57.580216 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:57.580263 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.580265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.580941 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.580942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.581556 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.581558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.581944 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.581945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.582008 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.582519 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:57.582524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:57.584481 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
W20260812 14:47:57.584636 31582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:57.579580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:34171 (local address 127.13.146.190:35081)
0812 14:47:57.579633 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:57.579636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:57.579818 (+ 182us) server_negotiation.cc:407] Connection header received
0812 14:47:57.579906 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:57.579907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:57.579921 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:57.579993 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:57.580345 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.580846 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.581648 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.581854 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.581943 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.584574 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:34171: BlockingRecv error: recv got EOF from 127.13.146.130:34171 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:57 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.131@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:57.737106 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:57.732238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.131:54493)
0812 14:47:57.732400 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:57.732409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:57.732491 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:57.732686 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:57.732687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:57.732898 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:57.732947 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.732949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.733711 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.733713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.734263 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:57.734265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.734649 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.734651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:57.734732 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.735212 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:57.735217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:57.737040 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
W20260812 14:47:57.737267 31582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:57.732280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:54493 (local address 127.13.146.190:35081)
0812 14:47:57.732335 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:57.732338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:57.732473 (+ 135us) server_negotiation.cc:407] Connection header received
0812 14:47:57.732548 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:57.732550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:57.732566 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:57.732668 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:57.733047 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.733620 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.734358 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:57.734567 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:57.734649 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:57.737140 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.131:54493: BlockingRecv error: recv got EOF from 127.13.146.131:54493 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 14:47:58 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:58 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.129@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:58.454512 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:58.444868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.129:49399)
0812 14:47:58.445041 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:58.445052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:58.445139 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:58.445316 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:58.445316 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:58.445484 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:58.445537 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:58.445539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.446229 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.446230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:58.446765 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:58.446766 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.447126 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.447127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:58.447167 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:58.452231 (+ 5064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:58.452237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:58.454412 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
W20260812 14:47:58.454576 31592 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:58.444954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:49399 (local address 127.13.146.190:35081)
0812 14:47:58.445093 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:58.445097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:58.445127 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:47:58.445200 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:58.445201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:58.445234 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:58.445299 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:58.445601 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.446153 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.446846 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.447040 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.447095 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:58.454504 (+ 7409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.129:49399: BlockingRecv error: recv got EOF from 127.13.146.129:49399 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
Aug 12 14:47:58 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:47:58 dist-test-slave-8hhm krb5kdc[29625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.146.130@KRBTEST.COM for oryx/127.13.146.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:47:58.589776 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:58.585161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:35081 (local address 127.13.146.130:35965)
0812 14:47:58.585411 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:58.585421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:58.585482 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:58.585615 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:58.585616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:58.585786 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:58.585837 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:58.585839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.586451 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.586451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:58.586984 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:58.586985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.587304 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.587304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:58.587344 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:58.587797 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:47:58.587802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:58.589664 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.146.190:35081: Server oryx/127.13.146.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":186,"threads_started":1}
W20260812 14:47:58.589824 31592 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:58.585276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:35965 (local address 127.13.146.190:35081)
0812 14:47:58.585359 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:58.585361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:58.585483 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:47:58.585531 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:58.585531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:58.585544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:58.585593 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:58.585900 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.586383 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.587056 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.587240 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.587290 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:58.589757 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.146.130:35965: BlockingRecv error: recv got EOF from 127.13.146.130:35965 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:47:58.660593 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 30747
I20260812 14:47:58.695555 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 31032
I20260812 14:47:58.708036 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 31247
I20260812 14:47:58.723773 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 30618
2026-08-12T14:47:58Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17923 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T14:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:58Z Disabled control of system clock
I20260812 14:47:58.765303 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34087
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:47:58.849349 31603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:58.849619 31603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:58.849658 31603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:58.850916 31603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:58.850994 31603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:58.851014 31603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:58.851034 31603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:58.851061 31603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:58.852464 31603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:58.852761 31603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:58.853076 31603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:58.858310 31620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:58.858333 31613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:58.858472 31603 server_base.cc:1034] running on GCE node
W20260812 14:47:58.858413 31612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:58.858803 31603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:58.859146 31603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:58.860354 31603 hybrid_clock.cc:648] HybridClock initialized: now 1786546078860334 us; error 37 us; skew 500 ppm
I20260812 14:47:58.861783 31603 webserver.cc:460] Webserver started at http://127.13.146.190:43695/ using document root <none> and password file <none>
I20260812 14:47:58.862056 31603 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:58.862130 31603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:58.862275 31603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:58.863395 31603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "9a50f46211db48a19e87ea6d54292ab9"
format_stamp: "Formatted at 2026-08-12 14:47:58 on dist-test-slave-8hhm"
server_key: "cbb02b8aff31496464d6f5cfc9de4390"
server_key_iv: "3b177c907a756217f2c56f80655b4ca3"
server_key_version: "encryptionkey@0"
I20260812 14:47:58.863811 31603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "9a50f46211db48a19e87ea6d54292ab9"
format_stamp: "Formatted at 2026-08-12 14:47:58 on dist-test-slave-8hhm"
server_key: "cbb02b8aff31496464d6f5cfc9de4390"
server_key_iv: "3b177c907a756217f2c56f80655b4ca3"
server_key_version: "encryptionkey@0"
I20260812 14:47:58.865191 31603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:47:58.865991 31631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:58.866164 31603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:47:58.866273 31603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "9a50f46211db48a19e87ea6d54292ab9"
format_stamp: "Formatted at 2026-08-12 14:47:58 on dist-test-slave-8hhm"
server_key: "cbb02b8aff31496464d6f5cfc9de4390"
server_key_iv: "3b177c907a756217f2c56f80655b4ca3"
server_key_version: "encryptionkey@0"
I20260812 14:47:58.866350 31603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:58.888149 31603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:58.888527 31603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:58.888700 31603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:58.892926 31603 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34087
I20260812 14:47:58.892964 31730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34087 every 8 connection(s)
I20260812 14:47:58.893356 31603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:58.894069 31732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:58.896834 31732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9: Bootstrap starting.
I20260812 14:47:58.897442 31732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:58.897689 31732 log.cc:824] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:58.898358 31732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9: No bootstrap required, opened a new log
I20260812 14:47:58.899883 31732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } }
I20260812 14:47:58.900050 31732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:58.900115 31732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a50f46211db48a19e87ea6d54292ab9, State: Initialized, Role: FOLLOWER
I20260812 14:47:58.900244 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 31603
I20260812 14:47:58.900341 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:58.900326 31732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } }
I20260812 14:47:58.900424 31732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:58.900468 31732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:58.900521 31732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:58.900552 13898 external_mini_cluster.cc:1428] Setting key e19a01a0d51b634e4efcdfe5e3f469ba
I20260812 14:47:58.901234 31732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } }
I20260812 14:47:58.901361 31732 leader_election.cc:302] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a50f46211db48a19e87ea6d54292ab9; no voters:
I20260812 14:47:58.901539 31732 leader_election.cc:288] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:58.901573 31743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:58.901749 31743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 9a50f46211db48a19e87ea6d54292ab9, State: Running, Role: LEADER
I20260812 14:47:58.901809 31732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:58.901901 31743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } }
I20260812 14:47:58.902384 31756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a50f46211db48a19e87ea6d54292ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } } }
I20260812 14:47:58.902395 31759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a50f46211db48a19e87ea6d54292ab9. Latest consensus state: current_term: 1 leader_uuid: "9a50f46211db48a19e87ea6d54292ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a50f46211db48a19e87ea6d54292ab9" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34087 } } }
I20260812 14:47:58.902555 31756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:58.902567 31759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:58.902990 31767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:58.903524 31767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:58.905231 31767 catalog_manager.cc:1347] Generated new cluster ID: 4d4ea552023543b8964363cc4862c1e0
I20260812 14:47:58.905282 31767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:58.907351 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:58.901105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.13.146.190:34087)
0812 14:47:58.901340 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:58.901349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:58.901383 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:47:58.901501 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:58.901510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:58.901568 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:58.901965 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:58.902096 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.903991 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.905043 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:58.905644 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:58.905756 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:58.906543 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:58.906547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:58.906547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:58.906755 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:58.906800 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:58.906814 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:58.906877 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:58.906982 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:47:58.907152 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":127,"threads_started":1}
I20260812 14:47:58.910904 31767 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:58.911407 31767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:58.916693 31767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a50f46211db48a19e87ea6d54292ab9: Generated new TSK 0
I20260812 14:47:58.916893 31767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:58.921193 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--builtin_ntp_servers=127.13.146.148:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 14:47:59.027446 31786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.027765 31786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.027812 31786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.029973 31786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.030035 31786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.030076 31786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:59.032431 31786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.032755 31786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.033159 31786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.039127 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.039341 31786 server_base.cc:1034] running on GCE node
W20260812 14:47:59.039417 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.042038 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.042357 31786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.042694 31786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.043869 31786 hybrid_clock.cc:648] HybridClock initialized: now 1786546079043844 us; error 36 us; skew 500 ppm
I20260812 14:47:59.045763 31786 webserver.cc:460] Webserver started at http://127.13.146.129:33749/ using document root <none> and password file <none>
I20260812 14:47:59.046058 31786 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.046136 31786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.046288 31786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.047932 31786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "c45730c728c04430992c8d7ea1f21a67"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "9739376f23773e567ee08c9df5568253"
server_key_iv: "bff44166c76ddaa88f7f53e1f7de346b"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.048445 31786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "c45730c728c04430992c8d7ea1f21a67"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "9739376f23773e567ee08c9df5568253"
server_key_iv: "bff44166c76ddaa88f7f53e1f7de346b"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.050383 31786 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260812 14:47:59.051548 31814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.051800 31786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:59.051910 31786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "c45730c728c04430992c8d7ea1f21a67"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "9739376f23773e567ee08c9df5568253"
server_key_iv: "bff44166c76ddaa88f7f53e1f7de346b"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.052062 31786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.084445 31786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.084753 31786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.084860 31786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.085075 31786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.085491 31786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.085533 31786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.085557 31786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.085570 31786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.091507 31786 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:45685
I20260812 14:47:59.091619 32037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:45685 every 8 connection(s)
I20260812 14:47:59.091961 31786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:59.095753 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 31786
I20260812 14:47:59.095852 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:59.096050 13898 external_mini_cluster.cc:1428] Setting key bd131d45095d147c54caa6b7df7ca879
I20260812 14:47:59.097101 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--builtin_ntp_servers=127.13.146.148:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 14:47:59.098117 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.092856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:36313 (local address 127.13.146.190:34087)
0812 14:47:59.092929 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.092931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.093168 (+ 237us) server_negotiation.cc:407] Connection header received
0812 14:47:59.093729 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.093730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.093744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.093815 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.094509 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.095018 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.096414 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.096659 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.096730 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.097754 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.097756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.097756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.097765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.097783 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.097788 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.097850 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.097924 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.097959 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:47:59.098237 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.092788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34087 (local address 127.13.146.129:36313)
0812 14:47:59.093008 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.093076 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.093607 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.093874 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.093878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.093887 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.094403 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.094425 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.095130 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.095131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.096326 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.096329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.096775 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.096777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.096934 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.097597 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.097598 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.097629 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.097653 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.097855 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.097860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.097863 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.097916 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.098000 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260812 14:47:59.098732 32038 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34087
I20260812 14:47:59.098855 32038 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.099058 32038 heartbeater.cc:511] Master 127.13.146.190:34087 requested a full tablet report, sending...
I20260812 14:47:59.099598 31662 ts_manager.cc:194] Registered new tserver with Master: c45730c728c04430992c8d7ea1f21a67 (127.13.146.129:45685)
I20260812 14:47:59.100206 31662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.129:36313
W20260812 14:47:59.183748 32049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.183971 32049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.183995 32049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.185145 32049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.185181 32049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.185257 32049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:59.186610 32049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.186822 32049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.187156 32049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.192531 32059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.192571 32060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.192654 32049 server_base.cc:1034] running on GCE node
W20260812 14:47:59.192584 32062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.192996 32049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.193347 32049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.194521 32049 hybrid_clock.cc:648] HybridClock initialized: now 1786546079194510 us; error 31 us; skew 500 ppm
I20260812 14:47:59.195914 32049 webserver.cc:460] Webserver started at http://127.13.146.130:35893/ using document root <none> and password file <none>
I20260812 14:47:59.196168 32049 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.196244 32049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.196396 32049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.197679 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "36f93851c0274294807b6f276f6ef4f9"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "721766dcddb1e551415727c55e080f76"
server_key_iv: "12cbeb09c498dbe97672d834828198e5"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.198064 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "36f93851c0274294807b6f276f6ef4f9"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "721766dcddb1e551415727c55e080f76"
server_key_iv: "12cbeb09c498dbe97672d834828198e5"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.199393 32049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:47:59.200095 32071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.200286 32049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:47:59.200398 32049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "36f93851c0274294807b6f276f6ef4f9"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "721766dcddb1e551415727c55e080f76"
server_key_iv: "12cbeb09c498dbe97672d834828198e5"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.200476 32049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.220220 32049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.220562 32049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.220731 32049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.220981 32049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.221405 32049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.221472 32049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.221515 32049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.221545 32049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.227466 32049 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34123
I20260812 14:47:59.227514 32279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34123 every 8 connection(s)
I20260812 14:47:59.227854 32049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:59.233071 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.228659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:54587 (local address 127.13.146.190:34087)
0812 14:47:59.228731 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.228734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.228990 (+ 256us) server_negotiation.cc:407] Connection header received
0812 14:47:59.229571 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.229572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.229584 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.229634 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.230296 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.230773 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.231757 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.231943 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.231998 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.232824 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.232825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.232825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.232829 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.232844 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.232848 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.232903 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.232987 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.233010 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:47:59.233269 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.228656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34087 (local address 127.13.146.130:54587)
0812 14:47:59.228865 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.228900 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.229479 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.229696 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.229700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.229715 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.230223 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.230228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.230857 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.230858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.231664 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.231666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.232023 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.232024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.232133 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.232663 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.232664 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.232692 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.232726 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.232913 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.232917 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.232920 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.232972 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.233053 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260812 14:47:59.233830 32280 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34087
I20260812 14:47:59.233937 32280 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.233982 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 32049
I20260812 14:47:59.234067 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:59.234133 32280 heartbeater.cc:511] Master 127.13.146.190:34087 requested a full tablet report, sending...
I20260812 14:47:59.234378 13898 external_mini_cluster.cc:1428] Setting key 583d4cf6f79bcf7b6b7d0def7422255c
I20260812 14:47:59.234611 31662 ts_manager.cc:194] Registered new tserver with Master: 36f93851c0274294807b6f276f6ef4f9 (127.13.146.130:34123)
I20260812 14:47:59.235262 31662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.130:54587
I20260812 14:47:59.235630 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--builtin_ntp_servers=127.13.146.148:45619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 14:47:59.319744 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.319985 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.320022 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.321158 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.321250 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.321296 32287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:59.322635 32287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.322914 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.323212 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.328323 32309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.328323 32305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.328346 32301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.328802 32287 server_base.cc:1034] running on GCE node
I20260812 14:47:59.329031 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.329339 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.330497 32287 hybrid_clock.cc:648] HybridClock initialized: now 1786546079330491 us; error 40 us; skew 500 ppm
I20260812 14:47:59.331681 32287 webserver.cc:460] Webserver started at http://127.13.146.131:44501/ using document root <none> and password file <none>
I20260812 14:47:59.331954 32287 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.332023 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.332185 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.333341 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "b931a4a26d934e9a8c7adaa6a47ea798"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "52159e2ed72377793d36e8439b9d2819"
server_key_iv: "b50f04ea792c75647f71ee382764ecbc"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.333729 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "b931a4a26d934e9a8c7adaa6a47ea798"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "52159e2ed72377793d36e8439b9d2819"
server_key_iv: "b50f04ea792c75647f71ee382764ecbc"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.335114 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:47:59.335922 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.336131 32287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:59.336243 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "b931a4a26d934e9a8c7adaa6a47ea798"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "52159e2ed72377793d36e8439b9d2819"
server_key_iv: "b50f04ea792c75647f71ee382764ecbc"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.336320 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.347373 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.347733 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.347893 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.348140 32287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.348479 32287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.348537 32287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.348584 32287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.348608 32287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.354825 32287 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:42917
I20260812 14:47:59.354866 32521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:42917 every 8 connection(s)
I20260812 14:47:59.355283 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:59.361304 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.356068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47813 (local address 127.13.146.190:34087)
0812 14:47:59.356196 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.356199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.356428 (+ 229us) server_negotiation.cc:407] Connection header received
0812 14:47:59.357117 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.357118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.357131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.357200 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.358167 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.358805 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.359955 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.360137 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.360213 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.361003 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.361004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.361004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.361009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.361023 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.361027 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.361075 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.361126 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.361178 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:47:59.361418 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 32287
I20260812 14:47:59.361521 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:59.361515 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.356043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34087 (local address 127.13.146.131:47813)
0812 14:47:59.356277 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.356320 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.357006 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.357312 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.357316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.357327 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.358011 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.358018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.358880 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.358899 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.359875 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:47:59.359878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:47:59.360243 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:47:59.360244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:47:59.360383 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:47:59.360873 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.360874 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.360902 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.360924 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.361077 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.361079 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.361082 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.361127 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.361273 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260812 14:47:59.361773 13898 external_mini_cluster.cc:1428] Setting key 783fb404fd095d53171cc269b1b70233
I20260812 14:47:59.362015 32523 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34087
I20260812 14:47:59.362139 32523 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.362356 32523 heartbeater.cc:511] Master 127.13.146.190:34087 requested a full tablet report, sending...
I20260812 14:47:59.362783 31662 ts_manager.cc:194] Registered new tserver with Master: b931a4a26d934e9a8c7adaa6a47ea798 (127.13.146.131:42917)
I20260812 14:47:59.363178 31662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.131:47813
I20260812 14:47:59.373766 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:47:59.375808 31741 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:59.375171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.13.146.190:34087)
0812 14:47:59.375232 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.375233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.375393 (+ 160us) server_negotiation.cc:407] Connection header received
0812 14:47:59.375474 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.375475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.375490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.375558 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.375720 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35444: BlockingRecv error: recv got EOF from 127.0.0.1:35444 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:47:59.375839 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 31786
I20260812 14:47:59.396639 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 32049
I20260812 14:47:59.407938 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 32287
I20260812 14:47:59.418401 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 31603
2026-08-12T14:47:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (687 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T14:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:47:59Z Disabled control of system clock
I20260812 14:47:59.452633 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37333
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:47:59.536332 32546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.536550 32546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.536571 32546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.537827 32546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:47:59.537873 32546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.537885 32546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.537899 32546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:47:59.537910 32546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:47:59.539229 32546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:37333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.539426 32546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.539670 32546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.544757 32554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.544752 32553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.544770 32562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.544840 32546 server_base.cc:1034] running on GCE node
I20260812 14:47:59.545324 32546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.545636 32546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.546787 32546 hybrid_clock.cc:648] HybridClock initialized: now 1786546079546773 us; error 29 us; skew 500 ppm
I20260812 14:47:59.548161 32546 webserver.cc:460] Webserver started at http://127.13.146.190:39423/ using document root <none> and password file <none>
I20260812 14:47:59.548367 32546 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.548409 32546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.548501 32546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.549639 32546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "1dfff07694194c23b9913ee4639c9ba2"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "8e803c2a16a57ce43d55a8ab9b55a88d"
server_key_iv: "d55f8fa3a18bd0a4b053c1892f34df7c"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.549954 32546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "1dfff07694194c23b9913ee4639c9ba2"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "8e803c2a16a57ce43d55a8ab9b55a88d"
server_key_iv: "d55f8fa3a18bd0a4b053c1892f34df7c"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.551283 32546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:47:59.552119 32570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.552419 32546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:47:59.552542 32546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "1dfff07694194c23b9913ee4639c9ba2"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "8e803c2a16a57ce43d55a8ab9b55a88d"
server_key_iv: "d55f8fa3a18bd0a4b053c1892f34df7c"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.552627 32546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.556068 32546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.556370 32546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.556524 32546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.560662 32546 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:37333
I20260812 14:47:59.560711 32661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:37333 every 8 connection(s)
I20260812 14:47:59.561058 32546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:47:59.561849 32662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:47:59.564699 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: Bootstrap starting.
I20260812 14:47:59.565405 32662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: Neither blocks nor log segments found. Creating new log.
I20260812 14:47:59.565713 32662 log.cc:824] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:47:59.566416 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: No bootstrap required, opened a new log
I20260812 14:47:59.567337 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 32546
I20260812 14:47:59.567474 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:47:59.567682 13898 external_mini_cluster.cc:1428] Setting key a4aa16003c8f56ce177f8281b17f82a7
I20260812 14:47:59.568336 32662 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } }
I20260812 14:47:59.568540 32662 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:47:59.568594 32662 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfff07694194c23b9913ee4639c9ba2, State: Initialized, Role: FOLLOWER
I20260812 14:47:59.568753 32662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } }
I20260812 14:47:59.568838 32662 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:47:59.568874 32662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:47:59.568964 32662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:47:59.569770 32662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } }
I20260812 14:47:59.569893 32662 leader_election.cc:302] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dfff07694194c23b9913ee4639c9ba2; no voters:
I20260812 14:47:59.570084 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.568271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.13.146.190:37333)
0812 14:47:59.568500 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.568508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.568533 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:47:59.568668 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.568676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.568719 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.568852 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.569552 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.569556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.569557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.569561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.569597 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.569607 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.569663 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.569765 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.569838 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260812 14:47:59.570097 32662 leader_election.cc:288] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:47:59.570389 32671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:47:59.570420 32662 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:47:59.570892 32671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 1dfff07694194c23b9913ee4639c9ba2, State: Running, Role: LEADER
I20260812 14:47:59.571090 32671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } }
I20260812 14:47:59.571533 32673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dfff07694194c23b9913ee4639c9ba2. Latest consensus state: current_term: 1 leader_uuid: "1dfff07694194c23b9913ee4639c9ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } } }
I20260812 14:47:59.571664 32673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [sys.catalog]: This master's current role is: LEADER
I20260812 14:47:59.571686 32672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dfff07694194c23b9913ee4639c9ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfff07694194c23b9913ee4639c9ba2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37333 } } }
I20260812 14:47:59.571758 32672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2 [sys.catalog]: This master's current role is: LEADER
W20260812 14:47:59.572304 32698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:47:59.572358 32698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:47:59.572410 32697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:47:59.572574 32697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:47:59.574199 32697 catalog_manager.cc:1347] Generated new cluster ID: 42ff1c5f65024c319ba4377c163fd5bb
I20260812 14:47:59.574258 32697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:47:59.603856 32697 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:47:59.604089 32697 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:47:59.609144 32697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dfff07694194c23b9913ee4639c9ba2: Generated new TSK 0
I20260812 14:47:59.609387 32697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:47:59.613549 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--builtin_ntp_servers=127.13.146.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:47:59.715476 32705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.715691 32705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.715713 32705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.716902 32705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.716944 32705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.716970 32705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:47:59.718407 32705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.718678 32705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.718998 32705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.725533 32717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.725528 32716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.725617 32705 server_base.cc:1034] running on GCE node
W20260812 14:47:59.725533 32720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.725911 32705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.726190 32705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.727366 32705 hybrid_clock.cc:648] HybridClock initialized: now 1786546079727354 us; error 29 us; skew 500 ppm
I20260812 14:47:59.728765 32705 webserver.cc:460] Webserver started at http://127.13.146.129:45497/ using document root <none> and password file <none>
I20260812 14:47:59.729043 32705 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.729099 32705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.729255 32705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.730769 32705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "40f1c3fe29714cd39f0628e6bf77d99a"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "887d9025f5113bc7451f6d8456889f59"
server_key_iv: "3483b0d5d3ea83edaa76ceadb3e7822d"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.731220 32705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "40f1c3fe29714cd39f0628e6bf77d99a"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "887d9025f5113bc7451f6d8456889f59"
server_key_iv: "3483b0d5d3ea83edaa76ceadb3e7822d"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.732864 32705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:47:59.733767 32736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.733963 32705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:47:59.734067 32705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "40f1c3fe29714cd39f0628e6bf77d99a"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "887d9025f5113bc7451f6d8456889f59"
server_key_iv: "3483b0d5d3ea83edaa76ceadb3e7822d"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.734146 32705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.737485 32705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.737756 32705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.737897 32705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.738103 32705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.738411 32705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.738444 32705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.738500 32705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.738525 32705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.744407 32705 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:34017
I20260812 14:47:59.744470 450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:34017 every 8 connection(s)
I20260812 14:47:59.744823 32705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:47:59.748072 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.745621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:40245 (local address 127.13.146.190:37333)
0812 14:47:59.745683 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.745685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.745938 (+ 253us) server_negotiation.cc:407] Connection header received
0812 14:47:59.746462 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.746463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.746472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.746542 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.747774 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.747775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.747776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.747780 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.747797 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.747802 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.747874 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.747967 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.747984 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:47:59.748278 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.745607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37333 (local address 127.13.146.129:40245)
0812 14:47:59.745812 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.745846 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.746359 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.746642 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.746646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.746655 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.747570 (+ 915us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.747570 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.747598 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.747627 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.747873 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.747881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.747884 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.747979 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.748040 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260812 14:47:59.748822 452 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37333
I20260812 14:47:59.748936 452 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.749087 452 heartbeater.cc:511] Master 127.13.146.190:37333 requested a full tablet report, sending...
I20260812 14:47:59.749696 32597 ts_manager.cc:194] Registered new tserver with Master: 40f1c3fe29714cd39f0628e6bf77d99a (127.13.146.129:34017)
I20260812 14:47:59.750161 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 32705
I20260812 14:47:59.750240 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:47:59.750438 13898 external_mini_cluster.cc:1428] Setting key a257ba0fdf3b11ed6f3547ae7ca2b573
I20260812 14:47:59.751315 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--builtin_ntp_servers=127.13.146.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:47:59.838749 455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.839016 455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.839059 455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.840293 455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.840376 455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.840425 455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:47:59.841935 455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.842242 455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.842602 455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.848016 478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.848147 455 server_base.cc:1034] running on GCE node
W20260812 14:47:59.848143 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.848085 465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.848578 455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.848853 455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.850010 455 hybrid_clock.cc:648] HybridClock initialized: now 1786546079849984 us; error 43 us; skew 500 ppm
I20260812 14:47:59.851377 455 webserver.cc:460] Webserver started at http://127.13.146.130:42841/ using document root <none> and password file <none>
I20260812 14:47:59.851660 455 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.851760 455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.851926 455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.853020 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "f5ee926ae4e24292a70914ed672e047f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "c5b4786ce6ac64370143c1df9bbd3677"
server_key_iv: "ac6b6f33a12d94ec99e5da92af00e392"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.853435 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "f5ee926ae4e24292a70914ed672e047f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "c5b4786ce6ac64370143c1df9bbd3677"
server_key_iv: "ac6b6f33a12d94ec99e5da92af00e392"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.854799 455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:59.855605 485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.855841 455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:59.855944 455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "f5ee926ae4e24292a70914ed672e047f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "c5b4786ce6ac64370143c1df9bbd3677"
server_key_iv: "ac6b6f33a12d94ec99e5da92af00e392"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.856025 455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.859488 455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.859755 455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.859897 455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.860107 455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.860422 455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.860477 455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.860520 455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.860546 455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.866451 455 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:44347
I20260812 14:47:59.866498 705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:44347 every 8 connection(s)
I20260812 14:47:59.866835 455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:47:59.867306 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 455
I20260812 14:47:59.867412 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:47:59.867645 13898 external_mini_cluster.cc:1428] Setting key ef9e5246cc864e1d2b69ebf5b1971c5d
I20260812 14:47:59.868799 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--builtin_ntp_servers=127.13.146.148:42735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 14:47:59.870229 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.867919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:47751 (local address 127.13.146.190:37333)
0812 14:47:59.867990 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.867992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.868003 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:47:59.868552 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.868553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.868561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.868663 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.869919 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.869921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.869921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.869927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.869949 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.869954 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.870021 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.870110 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.870126 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:47:59.870338 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.867579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37333 (local address 127.13.146.130:47751)
0812 14:47:59.867876 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.867925 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.868439 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.868722 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.868725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.868733 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.869760 (+ 1027us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.869761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.869789 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.869819 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.870029 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.870034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.870036 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.870097 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.870147 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
I20260812 14:47:59.871075 710 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37333
I20260812 14:47:59.871232 710 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.871407 710 heartbeater.cc:511] Master 127.13.146.190:37333 requested a full tablet report, sending...
I20260812 14:47:59.871830 32597 ts_manager.cc:194] Registered new tserver with Master: f5ee926ae4e24292a70914ed672e047f (127.13.146.130:44347)
W20260812 14:47:59.953192 713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:47:59.953461 713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:47:59.953483 713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:47:59.954607 713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:47:59.954644 713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:47:59.954669 713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:47:59.956084 713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:47:59.956317 713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:47:59.956569 713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:47:59.961680 723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.961688 719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:47:59.961695 720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:47:59.961825 713 server_base.cc:1034] running on GCE node
I20260812 14:47:59.962149 713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:47:59.962462 713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:47:59.965727 713 hybrid_clock.cc:648] HybridClock initialized: now 1786546079965697 us; error 38 us; skew 500 ppm
I20260812 14:47:59.967118 713 webserver.cc:460] Webserver started at http://127.13.146.131:46757/ using document root <none> and password file <none>
I20260812 14:47:59.967374 713 fs_manager.cc:359] Metadata directory not provided
I20260812 14:47:59.967425 713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:47:59.967614 713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:47:59.968747 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "08f9e03835464b9dbd41a3f9e46b084f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "5eb46a62ba3e9dd351579073f2fc09a8"
server_key_iv: "2e4d40b0e558c3957b6616f27b9b9e51"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.969128 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "08f9e03835464b9dbd41a3f9e46b084f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "5eb46a62ba3e9dd351579073f2fc09a8"
server_key_iv: "2e4d40b0e558c3957b6616f27b9b9e51"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.970753 713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:59.971644 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.971864 713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:47:59.971973 713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "08f9e03835464b9dbd41a3f9e46b084f"
format_stamp: "Formatted at 2026-08-12 14:47:59 on dist-test-slave-8hhm"
server_key: "5eb46a62ba3e9dd351579073f2fc09a8"
server_key_iv: "2e4d40b0e558c3957b6616f27b9b9e51"
server_key_version: "encryptionkey@0"
I20260812 14:47:59.972061 713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:47:59.975625 713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:47:59.975911 713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:47:59.976055 713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:47:59.976275 713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:47:59.976617 713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:47:59.976675 713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.976717 713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:47:59.976743 713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:47:59.982614 713 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:45517
I20260812 14:47:59.982646 946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:45517 every 8 connection(s)
I20260812 14:47:59.982980 713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:47:59.985425 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 713
I20260812 14:47:59.985548 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:47:59.985824 13898 external_mini_cluster.cc:1428] Setting key 749e40489014b7f97b7dba59d8d62382
I20260812 14:47:59.986387 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.983755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47315 (local address 127.13.146.190:37333)
0812 14:47:59.983801 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.983803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.984023 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:47:59.984485 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.984486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.984492 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.984548 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.986094 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:47:59.986095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:47:59.986096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:47:59.986101 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:47:59.986119 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:47:59.986124 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:47:59.986182 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:47:59.986293 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:47:59.986311 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:47:59.986610 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:47:59.983714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37333 (local address 127.13.146.131:47315)
0812 14:47:59.983896 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:47:59.983931 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:47:59.984399 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:47:59.984607 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:47:59.984610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:47:59.984618 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:47:59.985905 (+ 1287us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:47:59.985905 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:47:59.985947 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:47:59.985985 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:47:59.986198 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:47:59.986204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:47:59.986207 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:47:59.986295 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:47:59.986375 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260812 14:47:59.987092 947 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37333
I20260812 14:47:59.987174 947 heartbeater.cc:464] Registering TS with master...
I20260812 14:47:59.987287 947 heartbeater.cc:511] Master 127.13.146.190:37333 requested a full tablet report, sending...
I20260812 14:47:59.987594 32597 ts_manager.cc:194] Registered new tserver with Master: 08f9e03835464b9dbd41a3f9e46b084f (127.13.146.131:45517)
I20260812 14:47:59.997864 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:47:59.999866 32670 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:47:59.999382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.13.146.190:37333)
0812 14:47:59.999431 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:47:59.999433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:47:59.999534 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:47:59.999583 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:47:59.999584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:47:59.999588 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:47:59.999636 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:47:59.999777 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45130: BlockingRecv error: recv got EOF from 127.0.0.1:45130 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:47:59.999969 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 32705
I20260812 14:48:00.025345 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 455
I20260812 14:48:00.041420 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 713
I20260812 14:48:00.060942 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 32546
2026-08-12T14:48:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (644 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:48:00 dist-test-slave-8hhm krb5kdc[960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:48:00 dist-test-slave-8hhm krb5kdc[960](info): set up 2 sockets
Aug 12 14:48:00 dist-test-slave-8hhm krb5kdc[960](info): commencing operation
krb5kdc: starting...
W20260812 14:48:01.390653 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:48:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:48:01.518216 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39925
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:39925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:01.603032 977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:01.603255 977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:01.603279 977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:01.604460 977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:48:01.604506 977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:01.604519 977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:01.604532 977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:48:01.604544 977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:48:01.606104 977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:39925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:01.606379 977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:01.606623 977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:01.611683 1003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:01.611703 998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:01.612011 977 server_base.cc:1034] running on GCE node
W20260812 14:48:01.612043 997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:01.612372 977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:01.612687 977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:01.613858 977 hybrid_clock.cc:648] HybridClock initialized: now 1786546081613826 us; error 55 us; skew 500 ppm
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:01.616487 977 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:48:01.616956 977 webserver.cc:460] Webserver started at http://127.13.146.190:36363/ using document root <none> and password file <none>
I20260812 14:48:01.617172 977 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:01.617282 977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:01.617420 977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:48:01.618553 977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "30f50007076b43f3b0942962275b8c91"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "1ac0dadbb99fab5307a2672c46eaa918"
server_key_iv: "55f08b1b36cd48f56c7f2733500a94df"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.618950 977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "30f50007076b43f3b0942962275b8c91"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "1ac0dadbb99fab5307a2672c46eaa918"
server_key_iv: "55f08b1b36cd48f56c7f2733500a94df"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.620362 977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:48:01.621125 1017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:01.621343 977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:48:01.621443 977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "30f50007076b43f3b0942962275b8c91"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "1ac0dadbb99fab5307a2672c46eaa918"
server_key_iv: "55f08b1b36cd48f56c7f2733500a94df"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.621539 977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:01.635361 977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:01.637511 977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:01.637665 977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:01.641999 977 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:39925
I20260812 14:48:01.642083 1118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:39925 every 8 connection(s)
I20260812 14:48:01.642431 977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:48:01.643503 1120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:01.644074 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 977
I20260812 14:48:01.644155 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:48:01.644353 13898 external_mini_cluster.cc:1428] Setting key 30eaf0f193b581792d884d066cc08332
I20260812 14:48:01.646720 1120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: Bootstrap starting.
I20260812 14:48:01.647480 1120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:01.647814 1120 log.cc:824] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:01.648583 1120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: No bootstrap required, opened a new log
I20260812 14:48:01.651010 1120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } }
I20260812 14:48:01.651192 1120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:01.651247 1120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f50007076b43f3b0942962275b8c91, State: Initialized, Role: FOLLOWER
I20260812 14:48:01.651405 1120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } }
I20260812 14:48:01.651510 1120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:01.651575 1120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:48:01.651635 1120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:01.652423 1120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } }
I20260812 14:48:01.652549 1120 leader_election.cc:302] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30f50007076b43f3b0942962275b8c91; no voters:
I20260812 14:48:01.652776 1120 leader_election.cc:288] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:48:01.652833 1126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:48:01.653015 1126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [term 1 LEADER]: Becoming Leader. State: Replica: 30f50007076b43f3b0942962275b8c91, State: Running, Role: LEADER
I20260812 14:48:01.653147 1120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:48:01.653177 1126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } }
I20260812 14:48:01.653582 1127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30f50007076b43f3b0942962275b8c91. Latest consensus state: current_term: 1 leader_uuid: "30f50007076b43f3b0942962275b8c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } } }
I20260812 14:48:01.653682 1127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [sys.catalog]: This master's current role is: LEADER
I20260812 14:48:01.653944 1132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:48:01.654522 1132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:48:01.654976 1128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30f50007076b43f3b0942962275b8c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f50007076b43f3b0942962275b8c91" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39925 } } }
I20260812 14:48:01.655215 1128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91 [sys.catalog]: This master's current role is: LEADER
I20260812 14:48:01.656383 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:01.644951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.13.146.190:39925)
0812 14:48:01.645147 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:01.645158 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:01.645195 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:48:01.645782 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:01.645791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:01.645853 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:01.646009 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:01.646589 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.647970 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.648830 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.649455 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.649585 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:01.652147 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:01.652151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:01.652152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:01.652163 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:01.654509 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:01.654986 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:01.654990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:01.654990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:01.655109 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:01.655358 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:01.655359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:01.655360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:01.655693 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:01.655814 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:01.656004 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 14:48:01.656145 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260812 14:48:01.657285 1132 catalog_manager.cc:1347] Generated new cluster ID: cbf4d994a7254b689b723c9055b9be69
I20260812 14:48:01.657331 1132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:48:01.679175 1132 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:48:01.679883 1132 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:48:01.688688 1132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: Generated new TSK 0
I20260812 14:48:01.688910 1132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:48:01.718436 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:01.805755 1159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:01.805977 1159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:01.806003 1159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:01.807236 1159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:01.807286 1159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:01.807312 1159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:48:01.809031 1159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:01.809334 1159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:01.809593 1159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:01.814992 1168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:01.815135 1159 server_base.cc:1034] running on GCE node
W20260812 14:48:01.814980 1165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:01.814980 1166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:01.815532 1159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:01.815809 1159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:01.816979 1159 hybrid_clock.cc:648] HybridClock initialized: now 1786546081816959 us; error 52 us; skew 500 ppm
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:01.819666 1159 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:48:01.820127 1159 webserver.cc:460] Webserver started at http://127.13.146.129:36129/ using document root <none> and password file <none>
I20260812 14:48:01.820407 1159 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:01.820478 1159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:01.820623 1159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:48:01.821774 1159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "a2b7f341666f4a70897e0f4349e86250"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "00bc679bb7062eab3d34f8498af8c667"
server_key_iv: "d92cc9996d1f1dbc07dd0a28daf62a96"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.822142 1159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "a2b7f341666f4a70897e0f4349e86250"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "00bc679bb7062eab3d34f8498af8c667"
server_key_iv: "d92cc9996d1f1dbc07dd0a28daf62a96"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.823550 1159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:48:01.824481 1179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:01.824677 1159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:48:01.824798 1159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "a2b7f341666f4a70897e0f4349e86250"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "00bc679bb7062eab3d34f8498af8c667"
server_key_iv: "d92cc9996d1f1dbc07dd0a28daf62a96"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.824877 1159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:01.836926 1159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:01.839461 1159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:01.839658 1159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:01.839917 1159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:01.840265 1159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:01.840322 1159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:01.840372 1159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:01.840399 1159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:01.846585 1159 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:44589
I20260812 14:48:01.846637 1388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:44589 every 8 connection(s)
I20260812 14:48:01.846984 1159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:01.854831 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 1159
I20260812 14:48:01.855255 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:48:01.855507 13898 external_mini_cluster.cc:1428] Setting key 2a964db19d2c0481171ed263a0d2ec4d
I20260812 14:48:01.857520 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:01.847797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:42945 (local address 127.13.146.190:39925)
0812 14:48:01.847866 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:01.847868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:01.848121 (+ 253us) server_negotiation.cc:407] Connection header received
0812 14:48:01.848659 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:01.848660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:01.848671 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:01.848770 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:01.849762 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.850276 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.851284 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.851479 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.851529 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:01.854025 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:01.854026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:01.854026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:01.854033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:01.855845 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:01.856404 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:01.856405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:01.856406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:01.856479 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:01.856743 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:01.856744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:01.856744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:01.856899 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:01.856972 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:01.857277 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 14:48:01.857383 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:48:01.857605 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:01.847761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39925 (local address 127.13.146.129:42945)
0812 14:48:01.847989 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:01.848032 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:01.848573 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:01.848837 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:01.848840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:01.849466 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:01.849678 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:01.849684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.850366 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.850367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:01.851207 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:01.851209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:01.851565 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:01.851566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:01.851682 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:01.852208 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:01.852217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:01.853940 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:01.855937 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:01.855946 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:01.855948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:01.856307 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:01.856546 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:01.856546 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:01.856547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:01.856661 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:01.856969 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:01.856973 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:01.857124 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 14:48:01.857260 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 14:48:01.857408 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260812 14:48:01.858497 1389 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39925
I20260812 14:48:01.858601 1389 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:01.858817 1389 heartbeater.cc:511] Master 127.13.146.190:39925 requested a full tablet report, sending...
I20260812 14:48:01.859350 1047 ts_manager.cc:194] Registered new tserver with Master: a2b7f341666f4a70897e0f4349e86250 (127.13.146.129:44589)
I20260812 14:48:01.859973 1047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:42945
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:48:01.884251 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:01.975028 1396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:01.975322 1396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:01.975369 1396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:01.976603 1396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:01.976687 1396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:01.976737 1396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:48:01.978397 1396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:01.978716 1396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:01.978986 1396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:01.984472 1408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:01.984481 1412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:01.984467 1407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:01.984763 1396 server_base.cc:1034] running on GCE node
I20260812 14:48:01.985008 1396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:01.985370 1396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:01.986534 1396 hybrid_clock.cc:648] HybridClock initialized: now 1786546081986517 us; error 32 us; skew 500 ppm
Aug 12 14:48:01 dist-test-slave-8hhm krb5kdc[960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:01.989375 1396 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:48:01.989957 1396 webserver.cc:460] Webserver started at http://127.13.146.130:42641/ using document root <none> and password file <none>
I20260812 14:48:01.990280 1396 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:01.990355 1396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:01.990509 1396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:48:01.991762 1396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "6a3ff96989c845febf81ff2ef1aff01c"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "ac91dfcabbe45f67788d0777296ac19e"
server_key_iv: "109bcb05b082d4823d087f53e5b003f0"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.992194 1396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "6a3ff96989c845febf81ff2ef1aff01c"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "ac91dfcabbe45f67788d0777296ac19e"
server_key_iv: "109bcb05b082d4823d087f53e5b003f0"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.993793 1396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 14:48:01.994680 1424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:01.994864 1396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:48:01.995018 1396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "6a3ff96989c845febf81ff2ef1aff01c"
format_stamp: "Formatted at 2026-08-12 14:48:01 on dist-test-slave-8hhm"
server_key: "ac91dfcabbe45f67788d0777296ac19e"
server_key_iv: "109bcb05b082d4823d087f53e5b003f0"
server_key_version: "encryptionkey@0"
I20260812 14:48:01.995110 1396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:02.011941 1396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:02.014660 1396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:02.014848 1396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:02.015146 1396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:02.015503 1396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:02.015563 1396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:02.015615 1396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:02.015650 1396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:02.022652 1396 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:38687
I20260812 14:48:02.022696 1648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:38687 every 8 connection(s)
I20260812 14:48:02.023095 1396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546081, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:02.030755 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 1396
I20260812 14:48:02.030889 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:48:02.031106 13898 external_mini_cluster.cc:1428] Setting key 86bbf5e091ce754d52a72d5d0340ebb4
I20260812 14:48:02.033695 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.024006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39925 (local address 127.13.146.130:37169)
0812 14:48:02.024197 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.024244 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.024876 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.025136 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.025139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.025800 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:02.026001 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.026006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.026718 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.026719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.027552 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.027554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.028019 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.028020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.028153 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.028660 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:02.028669 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:02.030183 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:02.032258 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:02.032265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:02.032267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:02.032507 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:02.032729 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:02.032730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:02.032730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:02.032839 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:02.033124 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:02.033129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:02.033305 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.033395 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.033523 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260812 14:48:02.033689 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.024038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:37169 (local address 127.13.146.190:39925)
0812 14:48:02.024102 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.024104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.024335 (+ 231us) server_negotiation.cc:407] Connection header received
0812 14:48:02.024986 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.024987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.025000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.025056 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:02.026110 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.026618 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.027682 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.027933 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.028008 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.030299 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:02.030301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:02.030302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:02.030310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:02.032172 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.032587 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.032588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.032589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.032658 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.032903 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.032904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.032905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.033057 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:02.033130 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.033444 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.033542 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:48:02.034531 1653 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39925
I20260812 14:48:02.034650 1653 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:02.034859 1653 heartbeater.cc:511] Master 127.13.146.190:39925 requested a full tablet report, sending...
I20260812 14:48:02.035332 1047 ts_manager.cc:194] Registered new tserver with Master: 6a3ff96989c845febf81ff2ef1aff01c (127.13.146.130:38687)
I20260812 14:48:02.035859 1047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:37169
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:48:02.060237 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:02.145437 1662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:02.145747 1662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:02.145778 1662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:02.146968 1662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:02.147018 1662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:02.147045 1662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:48:02.148591 1662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:02.148865 1662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:02.149144 1662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:02.154520 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:02.154675 1669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:02.154539 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:02.154846 1662 server_base.cc:1034] running on GCE node
I20260812 14:48:02.155062 1662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:02.155365 1662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:02.156555 1662 hybrid_clock.cc:648] HybridClock initialized: now 1786546082156519 us; error 46 us; skew 500 ppm
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:02.159358 1662 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:48:02.159768 1662 webserver.cc:460] Webserver started at http://127.13.146.131:43427/ using document root <none> and password file <none>
I20260812 14:48:02.159956 1662 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:02.159997 1662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:02.160095 1662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:48:02.161139 1662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "6100c886918a42a4b4acd237f6c1d8bf"
format_stamp: "Formatted at 2026-08-12 14:48:02 on dist-test-slave-8hhm"
server_key: "eadef8145c941ea84bb81b84e7308a6d"
server_key_iv: "de158980b6983149f5615d5f333523d2"
server_key_version: "encryptionkey@0"
I20260812 14:48:02.161548 1662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "6100c886918a42a4b4acd237f6c1d8bf"
format_stamp: "Formatted at 2026-08-12 14:48:02 on dist-test-slave-8hhm"
server_key: "eadef8145c941ea84bb81b84e7308a6d"
server_key_iv: "de158980b6983149f5615d5f333523d2"
server_key_version: "encryptionkey@0"
I20260812 14:48:02.162917 1662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:48:02.163654 1689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:02.163949 1662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:48:02.164065 1662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "6100c886918a42a4b4acd237f6c1d8bf"
format_stamp: "Formatted at 2026-08-12 14:48:02 on dist-test-slave-8hhm"
server_key: "eadef8145c941ea84bb81b84e7308a6d"
server_key_iv: "de158980b6983149f5615d5f333523d2"
server_key_version: "encryptionkey@0"
I20260812 14:48:02.164143 1662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:02.202203 1662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:02.204864 1662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:02.205042 1662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:02.205356 1662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:02.205715 1662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:02.205778 1662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:02.205823 1662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:02.205850 1662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:02.211824 1662 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36495
I20260812 14:48:02.211865 1904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36495 every 8 connection(s)
I20260812 14:48:02.212236 1662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:48:02.216883 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 1662
I20260812 14:48:02.217007 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:48:02.217264 13898 external_mini_cluster.cc:1428] Setting key c0f4d23e76be3482619231aecd1aa047
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546082, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:02.222764 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.213108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:41847 (local address 127.13.146.190:39925)
0812 14:48:02.213162 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.213164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.213411 (+ 247us) server_negotiation.cc:407] Connection header received
0812 14:48:02.213931 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.213932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.213945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.214001 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:02.214980 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.215504 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.216561 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.216898 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.216987 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.219673 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:02.219674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:02.219674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:02.219682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:02.221332 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.221822 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.221823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.221823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.221882 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.222135 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.222136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.222136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.222258 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:02.222311 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.222624 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.222695 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:48:02.222986 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.213096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39925 (local address 127.13.146.131:41847)
0812 14:48:02.213294 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.213334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.213840 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.214060 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.214063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.214673 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:02.214881 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.214887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.215614 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.215615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.216451 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.216454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.217049 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.217050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.217181 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.217955 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:02.217967 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:02.219579 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:02.221434 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:02.221442 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:02.221444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:02.221747 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:02.221951 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:02.221952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:02.221952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:02.222076 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:02.222339 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:02.222346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:02.222491 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.222600 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.222739 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260812 14:48:02.223526 1907 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39925
I20260812 14:48:02.223654 1907 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:02.223863 1907 heartbeater.cc:511] Master 127.13.146.190:39925 requested a full tablet report, sending...
I20260812 14:48:02.224261 1047 ts_manager.cc:194] Registered new tserver with Master: 6100c886918a42a4b4acd237f6c1d8bf (127.13.146.131:36495)
I20260812 14:48:02.224646 1047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:41847
I20260812 14:48:02.229285 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:48:02.237494 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.230678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.13.146.190:39925)
0812 14:48:02.230764 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.230766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.230870 (+ 104us) server_negotiation.cc:407] Connection header received
0812 14:48:02.230959 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.230960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.230973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.231037 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:02.231690 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.232270 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.233107 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.233397 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.233481 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.234838 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:02.234840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:02.234841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:02.234849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:02.236080 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.236469 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.236471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.236471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.236537 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:02.236789 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:02.236790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:02.236791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:02.236967 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:02.237089 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.237276 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.237374 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:48:02.239449 1047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:48:02.241143 1047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:48:02.248136 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.244276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.13.146.131:36495)
0812 14:48:02.244518 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.244522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.244532 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:48:02.244568 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.244570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.244586 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.244812 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:02.244871 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.245925 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247167 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247666 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247766 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.247824 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.248002 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.248044 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
I20260812 14:48:02.248142 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.244053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36495 (local address 127.0.0.1:53444)
0812 14:48:02.244271 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.244280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.244321 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.244684 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.244685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.244687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:02.244740 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.244741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.246042 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.246044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.247078 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.247079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247765 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.247875 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.247897 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.247985 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.248033 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":61,"threads_started":1}
I20260812 14:48:02.248167 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.243760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.13.146.129:44589)
0812 14:48:02.243949 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.243954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.244141 (+ 187us) server_negotiation.cc:407] Connection header received
0812 14:48:02.244266 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.244269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.244286 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.244348 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:02.244633 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.245798 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247010 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247707 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247763 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.247824 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.248036 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.248083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260812 14:48:02.248140 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.243872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:38687 (local address 127.0.0.1:52166)
0812 14:48:02.244017 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.244068 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.244143 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.244411 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.244415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.244418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:02.244517 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.244519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.245552 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.245553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.246860 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.246863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247815 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.247893 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.247903 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.247943 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.248014 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260812 14:48:02.248414 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.243993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.13.146.130:38687)
0812 14:48:02.244203 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.244208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.244220 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:48:02.244262 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.244264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.244280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.244609 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:02.244691 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.245450 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.246951 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247707 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.248193 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.248262 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.248290 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.248332 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260812 14:48:02.248430 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.243639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44589 (local address 127.0.0.1:50810)
0812 14:48:02.244017 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.244063 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.244168 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.244458 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.244459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.244462 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:02.244544 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.244547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.245919 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.245920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.246886 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.246889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.247880 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.247881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.247946 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.247949 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.248291 (+ 342us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.248334 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":67,"threads_started":1}
I20260812 14:48:02.249253 1514 tablet_service.cc:1467] Processing CreateTablet for tablet 62820959b00a4da4ae98eb52cfe077cc (DEFAULT_TABLE table=test-table [id=6a8239c08f1d4260a33a4f965202041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:02.249336 1280 tablet_service.cc:1467] Processing CreateTablet for tablet 62820959b00a4da4ae98eb52cfe077cc (DEFAULT_TABLE table=test-table [id=6a8239c08f1d4260a33a4f965202041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:02.249672 1280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62820959b00a4da4ae98eb52cfe077cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:02.249672 1514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62820959b00a4da4ae98eb52cfe077cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:02.249663 1789 tablet_service.cc:1467] Processing CreateTablet for tablet 62820959b00a4da4ae98eb52cfe077cc (DEFAULT_TABLE table=test-table [id=6a8239c08f1d4260a33a4f965202041f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:02.250034 1789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62820959b00a4da4ae98eb52cfe077cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:02.252956 1946 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Bootstrap starting.
I20260812 14:48:02.253167 1947 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Bootstrap starting.
I20260812 14:48:02.253461 1948 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Bootstrap starting.
I20260812 14:48:02.253921 1946 tablet_bootstrap.cc:654] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:02.253921 1947 tablet_bootstrap.cc:654] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:02.254175 1948 tablet_bootstrap.cc:654] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:02.254575 1946 log.cc:824] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:02.254575 1947 log.cc:824] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:02.254858 1948 log.cc:824] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:02.255573 1946 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: No bootstrap required, opened a new log
I20260812 14:48:02.255652 1946 ts_tablet_manager.cc:1397] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:48:02.255800 1948 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: No bootstrap required, opened a new log
I20260812 14:48:02.255919 1948 ts_tablet_manager.cc:1397] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:48:02.257189 1946 raft_consensus.cc:348] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.257448 1946 raft_consensus.cc:374] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:02.257501 1946 raft_consensus.cc:729] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6100c886918a42a4b4acd237f6c1d8bf, State: Initialized, Role: FOLLOWER
I20260812 14:48:02.257511 1948 raft_consensus.cc:348] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.257633 1946 consensus_queue.cc:260] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.257675 1948 raft_consensus.cc:374] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:02.257736 1948 raft_consensus.cc:729] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3ff96989c845febf81ff2ef1aff01c, State: Initialized, Role: FOLLOWER
I20260812 14:48:02.257850 1948 consensus_queue.cc:260] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.257949 1946 ts_tablet_manager.cc:1428] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:48:02.258168 1653 heartbeater.cc:503] Master 127.13.146.190:39925 was elected leader, sending a full tablet report...
I20260812 14:48:02.258410 1947 tablet_bootstrap.cc:492] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: No bootstrap required, opened a new log
I20260812 14:48:02.258431 1948 ts_tablet_manager.cc:1428] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:48:02.258507 1947 ts_tablet_manager.cc:1397] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 14:48:02.260334 1947 raft_consensus.cc:348] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.260507 1947 raft_consensus.cc:374] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:02.260561 1947 raft_consensus.cc:729] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b7f341666f4a70897e0f4349e86250, State: Initialized, Role: FOLLOWER
I20260812 14:48:02.260694 1947 consensus_queue.cc:260] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.261015 1947 ts_tablet_manager.cc:1428] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:48:02.261041 1389 heartbeater.cc:503] Master 127.13.146.190:39925 was elected leader, sending a full tablet report...
I20260812 14:48:02.257992 1907 heartbeater.cc:503] Master 127.13.146.190:39925 was elected leader, sending a full tablet report...
W20260812 14:48:02.273733 1654 tablet.cc:2369] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:48:02.347805 1390 tablet.cc:2369] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:48:02.451669 1959 raft_consensus.cc:482] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:48:02.451860 1959 raft_consensus.cc:504] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.452400 1959 leader_election.cc:288] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b7f341666f4a70897e0f4349e86250 (127.13.146.129:44589), 6100c886918a42a4b4acd237f6c1d8bf (127.13.146.131:36495)
I20260812 14:48:02.455595 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.452614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44589 (local address 127.13.146.130:37599)
0812 14:48:02.452701 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.452717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.452792 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.452977 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.452978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.452979 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:02.453067 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.453070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.453815 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.453816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.454748 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.454751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.455314 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.455315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.455402 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.455405 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.455472 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.455514 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 14:48:02.455586 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.452721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:37599 (local address 127.13.146.129:44589)
0812 14:48:02.452803 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.452805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.452816 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:48:02.452895 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.452896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.452911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.452972 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:02.453150 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.453720 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.454845 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.455226 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.455317 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.455326 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.455480 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.455515 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 14:48:02.455686 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.452778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36495 (local address 127.13.146.130:46663)
0812 14:48:02.452973 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.452983 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:02.453038 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:02.453277 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:02.453278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:02.453280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:02.453356 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.453380 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.453993 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.453994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.454852 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:02.454854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.455411 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.455412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:02.455483 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.455486 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:48:02.455571 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:48:02.455606 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":106,"threads_started":1}
I20260812 14:48:02.455677 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.452864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:46663 (local address 127.13.146.131:36495)
0812 14:48:02.452922 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.452924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.453030 (+ 106us) server_negotiation.cc:407] Connection header received
0812 14:48:02.453151 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.453157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.453168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.453257 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:02.453455 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.453909 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.454942 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.455323 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.455411 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.455419 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.455566 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.455606 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:48:02.455996 1314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62820959b00a4da4ae98eb52cfe077cc" candidate_uuid: "6a3ff96989c845febf81ff2ef1aff01c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b7f341666f4a70897e0f4349e86250" is_pre_election: true
I20260812 14:48:02.456012 1824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62820959b00a4da4ae98eb52cfe077cc" candidate_uuid: "6a3ff96989c845febf81ff2ef1aff01c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6100c886918a42a4b4acd237f6c1d8bf" is_pre_election: true
I20260812 14:48:02.456156 1314 raft_consensus.cc:2454] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a3ff96989c845febf81ff2ef1aff01c in term 0.
I20260812 14:48:02.456225 1824 raft_consensus.cc:2454] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a3ff96989c845febf81ff2ef1aff01c in term 0.
I20260812 14:48:02.456437 1427 leader_election.cc:302] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3ff96989c845febf81ff2ef1aff01c, a2b7f341666f4a70897e0f4349e86250; no voters:
I20260812 14:48:02.456561 1959 raft_consensus.cc:2781] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:48:02.456622 1959 raft_consensus.cc:482] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:48:02.456650 1959 raft_consensus.cc:3037] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:02.457506 1959 raft_consensus.cc:504] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.457650 1959 leader_election.cc:288] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [CANDIDATE]: Term 1 election: Requested vote from peers a2b7f341666f4a70897e0f4349e86250 (127.13.146.129:44589), 6100c886918a42a4b4acd237f6c1d8bf (127.13.146.131:36495)
I20260812 14:48:02.457831 1314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62820959b00a4da4ae98eb52cfe077cc" candidate_uuid: "6a3ff96989c845febf81ff2ef1aff01c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b7f341666f4a70897e0f4349e86250"
I20260812 14:48:02.457861 1824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62820959b00a4da4ae98eb52cfe077cc" candidate_uuid: "6a3ff96989c845febf81ff2ef1aff01c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6100c886918a42a4b4acd237f6c1d8bf"
I20260812 14:48:02.457930 1824 raft_consensus.cc:3037] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:02.457926 1314 raft_consensus.cc:3037] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:02.458864 1314 raft_consensus.cc:2454] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3ff96989c845febf81ff2ef1aff01c in term 1.
I20260812 14:48:02.458864 1824 raft_consensus.cc:2454] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3ff96989c845febf81ff2ef1aff01c in term 1.
I20260812 14:48:02.459087 1427 leader_election.cc:302] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6100c886918a42a4b4acd237f6c1d8bf, 6a3ff96989c845febf81ff2ef1aff01c; no voters:
I20260812 14:48:02.459180 1959 raft_consensus.cc:2781] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:48:02.459347 1959 raft_consensus.cc:686] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 1 LEADER]: Becoming Leader. State: Replica: 6a3ff96989c845febf81ff2ef1aff01c, State: Running, Role: LEADER
I20260812 14:48:02.459471 1959 consensus_queue.cc:237] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } }
I20260812 14:48:02.460422 1046 catalog_manager.cc:5495] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a3ff96989c845febf81ff2ef1aff01c (127.13.146.130). New cstate: current_term: 1 leader_uuid: "6a3ff96989c845febf81ff2ef1aff01c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3ff96989c845febf81ff2ef1aff01c" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 38687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 } health_report { overall_health: UNKNOWN } } }
W20260812 14:48:02.462808 1910 tablet.cc:2369] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:48:02.479554 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.477157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.13.146.130:38687)
0812 14:48:02.477258 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:02.477260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:02.477277 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:48:02.477345 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:02.477346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:02.477365 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:02.477426 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:48:02.477603 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.478063 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.478821 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:02.479000 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:02.479050 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:02.479222 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:48:02.479318 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:48:02.479377 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:02.479408 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 14:48:02.479423 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:48:02.481513 1824 raft_consensus.cc:1261] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 1 FOLLOWER]: Refusing update from remote peer 6a3ff96989c845febf81ff2ef1aff01c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:48:02.481559 1314 raft_consensus.cc:1261] T 62820959b00a4da4ae98eb52cfe077cc P a2b7f341666f4a70897e0f4349e86250 [term 1 FOLLOWER]: Refusing update from remote peer 6a3ff96989c845febf81ff2ef1aff01c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:48:02.481918 1959 consensus_queue.cc:1035] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6100c886918a42a4b4acd237f6c1d8bf" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:48:02.482126 1962 consensus_queue.cc:1035] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b7f341666f4a70897e0f4349e86250" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:48:02.485410 1968 mvcc.cc:205] Tried to move back new op lower bound from 7317692753841422336 to 7317692753754517504. Current Snapshot: MvccSnapshot[applied={T|T < 7317692753841422336 or (T in {7317692753841422336})}]
I20260812 14:48:02.485332 1969 mvcc.cc:205] Tried to move back new op lower bound from 7317692753841422336 to 7317692753754517504. Current Snapshot: MvccSnapshot[applied={T|T < 7317692753841422336}]
I20260812 14:48:02.486122 1970 mvcc.cc:205] Tried to move back new op lower bound from 7317692753841422336 to 7317692753754517504. Current Snapshot: MvccSnapshot[applied={T|T < 7317692753841422336}]
I20260812 14:48:02.487241 1047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:48:02.487347 1047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:48:02.488137 1047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30f50007076b43f3b0942962275b8c91: Sending DeleteTablet for 3 replicas of tablet 62820959b00a4da4ae98eb52cfe077cc
I20260812 14:48:02.488484 1280 tablet_service.cc:1514] Processing DeleteTablet for tablet 62820959b00a4da4ae98eb52cfe077cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:02 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50810
I20260812 14:48:02.488502 1514 tablet_service.cc:1514] Processing DeleteTablet for tablet 62820959b00a4da4ae98eb52cfe077cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:02 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:52166
I20260812 14:48:02.488741 1789 tablet_service.cc:1514] Processing DeleteTablet for tablet 62820959b00a4da4ae98eb52cfe077cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:02 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:53444
I20260812 14:48:02.488971 1986 tablet_replica.cc:330] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: stopping tablet replica
I20260812 14:48:02.489079 1986 raft_consensus.cc:2229] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:48:02.489192 1986 raft_consensus.cc:2258] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:48:02.489259 1984 tablet_replica.cc:330] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: stopping tablet replica
I20260812 14:48:02.489368 1984 raft_consensus.cc:2229] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:48:02.489413 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 1159
I20260812 14:48:02.489502 1984 raft_consensus.cc:2258] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:48:02.498400 1984 ts_tablet_manager.cc:1905] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:48:02.500308 1984 ts_tablet_manager.cc:1918] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:48:02.500447 1984 log.cc:1196] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/62820959b00a4da4ae98eb52cfe077cc
I20260812 14:48:02.500676 1986 ts_tablet_manager.cc:1905] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:48:02.500753 1984 ts_tablet_manager.cc:1939] T 62820959b00a4da4ae98eb52cfe077cc P 6a3ff96989c845febf81ff2ef1aff01c: Deleting consensus metadata
I20260812 14:48:02.501262 1023 catalog_manager.cc:4841] TS 6a3ff96989c845febf81ff2ef1aff01c (127.13.146.130:38687): tablet 62820959b00a4da4ae98eb52cfe077cc (table test-table [id=6a8239c08f1d4260a33a4f965202041f]) successfully deleted
I20260812 14:48:02.502199 1986 ts_tablet_manager.cc:1918] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:48:02.502305 1986 log.cc:1196] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/62820959b00a4da4ae98eb52cfe077cc
I20260812 14:48:02.502568 1986 ts_tablet_manager.cc:1939] T 62820959b00a4da4ae98eb52cfe077cc P 6100c886918a42a4b4acd237f6c1d8bf: Deleting consensus metadata
I20260812 14:48:02.502995 1023 catalog_manager.cc:4841] TS 6100c886918a42a4b4acd237f6c1d8bf (127.13.146.131:36495): tablet 62820959b00a4da4ae98eb52cfe077cc (table test-table [id=6a8239c08f1d4260a33a4f965202041f]) successfully deleted
W20260812 14:48:02.515848 1023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:44589 (error 108)
I20260812 14:48:02.516078 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 1396
I20260812 14:48:02.522104 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:02.521921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:44589 (local address 127.0.0.1:50820)
0812 14:48:02.521995 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:02.522049 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 14:48:02.522394 1023 catalog_manager.cc:4584] TS a2b7f341666f4a70897e0f4349e86250 (127.13.146.129:44589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62820959b00a4da4ae98eb52cfe077cc: Network error: Client connection negotiation failed: client connection to 127.13.146.129:44589: connect: Connection refused (error 111)
I20260812 14:48:02.537703 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 1662
I20260812 14:48:02.552842 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 977
2026-08-12T14:48:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2491 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[1991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[1991](info): set up 2 sockets
Aug 12 14:48:02 dist-test-slave-8hhm krb5kdc[1991](info): commencing operation
krb5kdc: starting...
W20260812 14:48:03.867934 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:48:03 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:48:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangeradmin_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangeradmin_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangerlookup_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangerlookup_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
I20260812 14:48:04.049991 13898 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskS5QCDN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/postgres -l logfile start
2026-08-12 14:48:06.681 UTC [2058] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:48:06.681 UTC [2058] LOG: listening on IPv4 address "127.13.146.148", port 49735
2026-08-12 14:48:06.685 UTC [2058] LOG: listening on Unix socket "/tmp/.s.PGSQL.49735"
2026-08-12 14:48:06.708 UTC [2062] LOG: database system was shut down at 2026-08-12 14:48:05 UTC
2026-08-12 14:48:06.712 UTC [2058] LOG: database system is ready to accept connections
I20260812 14:48:07.930094 13898 mini_postgres.cc:96] Postgres bound to 49735
2026-08-12 14:48:07.938 UTC [2082] FATAL: database "slave" does not exist
127.13.146.148:49735 - accepting connections
I20260812 14:48:07.939335 13898 mini_ranger.cc:161] Starting Ranger...
I20260812 14:48:07.956429 13898 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:48:08.447551 13898 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:48:08.447645 13898 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin
2026-08-12 14:48:08,764 [I] DB FLAVOR :POSTGRES
2026-08-12 14:48:08,765 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:48:08,765 [I] Checking connection..
2026-08-12 14:48:08,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:48:09,033 [I] Checking connection passed.
2026-08-12 14:48:09,033 [I] --------- Verifying version history table ---------
2026-08-12 14:48:09,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:09,315 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:48:09,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:09,581 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:48:09,581 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:48:09,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:48:09.826 UTC [2201] WARNING: there is no transaction in progress
2026-08-12 14:48:09,836 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:48:09,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:10,086 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:48:10,086 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:48:10,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:48:10,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:48:10,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-12 14:48:10,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:48:11,134 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:48:11,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:48:11,382 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:48:11,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:48:11,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:48:11,915 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:48:11,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:48:12.889 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:12.906 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:12.922 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:12.937 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:12.953 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:12.971 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.026 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.033 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.047 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.058 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.069 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.083 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.092 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.099 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.106 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13.113 UTC [2397] WARNING: there is no transaction in progress
2026-08-12 14:48:13,769 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:48:13,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:48:14,033 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:48:14,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:48:14,284 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:48:14,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:48:14,542 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:48:14,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:48:14,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:48:15,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-12 14:48:15,361 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:48:15,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:48:15,601 [I] DB_PATCHES have already been applied
I20260812 14:48:15.607371 13898 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskS5QCDN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:48:15.607482 13898 mini_ranger.cc:193] Using host: 127.13.146.148
I20260812 14:48:15.611804 13898 mini_ranger.cc:227] Ranger admin URL: http://127.13.146.148:37157
Aug 12, 2026 2:48:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:48:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:48:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:48:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:48:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:48:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:48:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:48:17.104882 13898 mini_ranger.cc:160] Time spent starting Ranger: real 9.166s user 0.000s sys 0.013s
Aug 12, 2026 2:48:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:48:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 2:48:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:48:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:48:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:48:38.568508 13898 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangerkms_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/rangerkms_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:48:38 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546118, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:48:38.748585 13898 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:48:38.772670 13898 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:48:39.905000 13898 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:48:39.905112 13898 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms
I20260812 14:48:39.905133 13898 mini_ranger_kms.cc:227] Using postgres at 127.13.146.148:49735
2026-08-12 14:48:40,225 [I] DB FLAVOR :POSTGRES
2026-08-12 14:48:40,225 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:48:40,225 [I] Checking connection
2026-08-12 14:48:40,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:48:40,511 [I] connection success
2026-08-12 14:48:40,512 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:48:40,512 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:48:40,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:48:40,780 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:48:40,780 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:48:40,780 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:48:40,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:49735/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:48:41,077 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:48:41.547139 13898 mini_ranger_kms.cc:310] Created kms service
I20260812 14:48:41.736263 13898 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:48:41.782963 13898 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:48:41 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546118, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:42.207376 13898 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:48:42.207502 13898 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms:/tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:48:42.207526 13898 mini_ranger_kms.cc:240] Using host: 127.13.146.148
I20260812 14:48:42.211920 13898 mini_ranger_kms.cc:276] Ranger KMS PID: 2748
I20260812 14:48:42.212055 13898 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.146.148:55517
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:48:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 14:48:44 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786546124, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 14:48:45 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546124, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:45.255739 13898 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.507s user 0.000s sys 0.302s
I20260812 14:48:45.255846 13898 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:48:45.255919 13898 mini_ranger_kms.cc:401] 127.13.146.148:55517/kms/v1/keys
Aug 12 14:48:45 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546124, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
Aug 12 14:48:47 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:48:47.807819 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36333
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:39307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:36333
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:55517/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:47.895434 2835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:47.895658 2835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:47.895680 2835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:47.896973 2835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:48:47.897024 2835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:47.897039 2835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:47.897051 2835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:48:47.897063 2835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:48:47.898805 2835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:39307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.146.148:55517/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:36333
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:47.899065 2835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:47.899330 2835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:47.904428 2842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:47.904454 2851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:47.904444 2844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:47.904718 2835 server_base.cc:1034] running on GCE node
I20260812 14:48:47.905056 2835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:47.905438 2835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:47.906627 2835 hybrid_clock.cc:648] HybridClock initialized: now 1786546127906612 us; error 31 us; skew 500 ppm
Aug 12 14:48:47 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:47.913336 2835 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:48:47.913820 2835 webserver.cc:460] Webserver started at http://127.13.146.190:35677/ using document root <none> and password file <none>
I20260812 14:48:47.914091 2835 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:47.914155 2835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:47.914318 2835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:48:47 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546127, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:47.960063 2835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "a11345db35da4232ab1ce689eafe60b1"
format_stamp: "Formatted at 2026-08-12 14:48:47 on dist-test-slave-8hhm"
server_key: "ab47f7ae40e193f45d2326556d90131a"
server_key_iv: "149fe5f2e84085b7cb13d128e7a6563f"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:47.960502 2835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "a11345db35da4232ab1ce689eafe60b1"
format_stamp: "Formatted at 2026-08-12 14:48:47 on dist-test-slave-8hhm"
server_key: "ab47f7ae40e193f45d2326556d90131a"
server_key_iv: "149fe5f2e84085b7cb13d128e7a6563f"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:47.962203 2835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:48:47.973618 2863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:47.973881 2835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:48:47.974035 2835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a11345db35da4232ab1ce689eafe60b1"
format_stamp: "Formatted at 2026-08-12 14:48:47 on dist-test-slave-8hhm"
server_key: "ab47f7ae40e193f45d2326556d90131a"
server_key_iv: "149fe5f2e84085b7cb13d128e7a6563f"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:47.974223 2835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:47.987622 2835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:47.989836 2835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:47.989984 2835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:47.994915 2835 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:36333
I20260812 14:48:47.994966 2961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:36333 every 8 connection(s)
I20260812 14:48:47.995379 2835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:48:47.996148 2963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:47.998673 2963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: Bootstrap starting.
I20260812 14:48:47.999428 2963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:47.999727 2963 log.cc:824] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:48.000509 2963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: No bootstrap required, opened a new log
I20260812 14:48:48.002681 2963 raft_consensus.cc:348] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } }
I20260812 14:48:48.002864 2963 raft_consensus.cc:374] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:48.002918 2963 raft_consensus.cc:729] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a11345db35da4232ab1ce689eafe60b1, State: Initialized, Role: FOLLOWER
I20260812 14:48:48.003080 2963 consensus_queue.cc:260] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } }
I20260812 14:48:48.003166 2963 raft_consensus.cc:388] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:48:48.003218 2963 raft_consensus.cc:482] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:48:48.003268 2963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:48.003816 2963 raft_consensus.cc:504] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } }
I20260812 14:48:48.003940 2963 leader_election.cc:302] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a11345db35da4232ab1ce689eafe60b1; no voters:
I20260812 14:48:48.004149 2963 leader_election.cc:288] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:48:48.004179 2967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:48:48.004321 2967 raft_consensus.cc:686] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [term 1 LEADER]: Becoming Leader. State: Replica: a11345db35da4232ab1ce689eafe60b1, State: Running, Role: LEADER
I20260812 14:48:48.004513 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 2835
I20260812 14:48:48.004498 2967 consensus_queue.cc:237] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } }
I20260812 14:48:48.004623 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:48:48.004832 13898 external_mini_cluster.cc:1428] Setting key 816ddd846acbb9de77090c7f47ba3930
I20260812 14:48:48.004876 2967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a11345db35da4232ab1ce689eafe60b1. Latest consensus state: current_term: 1 leader_uuid: "a11345db35da4232ab1ce689eafe60b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } } }
I20260812 14:48:48.004954 2967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:48:48.004988 2969 sys_catalog.cc:455] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a11345db35da4232ab1ce689eafe60b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11345db35da4232ab1ce689eafe60b1" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36333 } } }
I20260812 14:48:48.005081 2969 sys_catalog.cc:458] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:48:48.005360 2963 sys_catalog.cc:564] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:48:48.005571 2977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:48:48.006117 2977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:48:48.007304 2963 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:48:48.007844 2977 catalog_manager.cc:1347] Generated new cluster ID: 75521a0446744109adb3c8a0a478a0ba
I20260812 14:48:48.007901 2977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:48:48 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:48.029193 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:48.005346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.13.146.190:36333)
0812 14:48:48.007224 (+ 1878us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:48.007233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:48.007267 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:48:48.007389 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:48.007397 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:48.007479 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:48.007633 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:48.009792 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:48.012094 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:48.021268 (+ 9174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:48.021975 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:48.022126 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:48.025245 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:48.025250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:48.025251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:48.025265 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:48.027422 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:48.027841 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:48.027844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:48.027845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:48.027954 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:48.028156 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:48.028157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:48.028157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:48.028494 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:48.028662 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:48.028805 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 14:48:48.028945 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1817,"thread_start_us":87,"threads_started":1}
I20260812 14:48:48.043186 2977 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:48:48.043912 2977 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:48:48.049350 2977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: Generated new TSK 0
I20260812 14:48:48.049546 2977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:48:48.824545 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:55517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:39307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:48.934841 3052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:48.935122 3052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:48.935155 3052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:48.936498 3052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:48.936543 3052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:48.936570 3052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:48:48.938282 3052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:39307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.146.148:55517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:48.938532 3052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:48.938802 3052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:48.945043 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:48.945055 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:48.945055 3061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:48.945174 3052 server_base.cc:1034] running on GCE node
I20260812 14:48:48.945688 3052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:48.945986 3052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:48.947153 3052 hybrid_clock.cc:648] HybridClock initialized: now 1786546128947138 us; error 31 us; skew 500 ppm
Aug 12 14:48:48 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546128, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:48.949761 3052 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:48:48.950309 3052 webserver.cc:460] Webserver started at http://127.13.146.129:38219/ using document root <none> and password file <none>
I20260812 14:48:48.950606 3052 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:48.950695 3052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:48.950843 3052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:48:48 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546128, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:48.968327 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "c8d2ba773b334c61b1d493141cfdcc3f"
format_stamp: "Formatted at 2026-08-12 14:48:48 on dist-test-slave-8hhm"
server_key: "0c20d5642ab80b466f19930f285173e7"
server_key_iv: "1f42567160350b60c1b1c6eaf4309322"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:48.968968 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "c8d2ba773b334c61b1d493141cfdcc3f"
format_stamp: "Formatted at 2026-08-12 14:48:48 on dist-test-slave-8hhm"
server_key: "0c20d5642ab80b466f19930f285173e7"
server_key_iv: "1f42567160350b60c1b1c6eaf4309322"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:48.977696 3052 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.001s
I20260812 14:48:48.994446 3085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:48.994642 3052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 14:48:48.994769 3052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "c8d2ba773b334c61b1d493141cfdcc3f"
format_stamp: "Formatted at 2026-08-12 14:48:48 on dist-test-slave-8hhm"
server_key: "0c20d5642ab80b466f19930f285173e7"
server_key_iv: "1f42567160350b60c1b1c6eaf4309322"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:48.994889 3052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:49.014294 3052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:49.016500 3052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:49.016674 3052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:49.016943 3052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:49.017357 3052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:49.017423 3052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.017468 3052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:49.017494 3052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.025254 3052 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:46175
I20260812 14:48:49.025658 3052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:48:49.026031 3277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:46175 every 8 connection(s)
I20260812 14:48:49.030534 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 3052
I20260812 14:48:49.030663 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:48:49.030885 13898 external_mini_cluster.cc:1428] Setting key 260aff4e0092216c4533b925027b59cd
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546128, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:49.036695 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.026480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:47307 (local address 127.13.146.190:36333)
0812 14:48:49.026690 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.026692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.027011 (+ 319us) server_negotiation.cc:407] Connection header received
0812 14:48:49.027612 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.027613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.027629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.027695 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:49.028277 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.028794 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.029872 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.030090 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.030182 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.033159 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:49.033160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:49.033161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:49.033168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:49.035109 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.035630 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.035631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.035631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.035702 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.035990 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.035991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.035991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.036131 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:49.036208 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.036501 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.036597 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260812 14:48:49.036855 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.026465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36333 (local address 127.13.146.129:47307)
0812 14:48:49.026889 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.026932 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.027489 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.027771 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.027775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.027981 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:49.028174 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.028178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.028903 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.028904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.029780 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.029782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.030193 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.030195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.030359 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.031028 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:49.031038 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:49.033053 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:49.035223 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.035231 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.035234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.035562 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.035788 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.035789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.035789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.035914 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.036199 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:49.036214 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:49.036360 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.036468 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.036618 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":360,"threads_started":1}
I20260812 14:48:49.037429 3278 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36333
I20260812 14:48:49.037528 3278 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:49.037717 3278 heartbeater.cc:511] Master 127.13.146.190:36333 requested a full tablet report, sending...
I20260812 14:48:49.038309 2896 ts_manager.cc:194] Registered new tserver with Master: c8d2ba773b334c61b1d493141cfdcc3f (127.13.146.129:46175)
I20260812 14:48:49.038992 2896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:47307
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:48:49.064023 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:55517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:39307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 14:48:49.194438 3285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:49.194741 3285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:49.194801 3285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:49.196525 3285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:49.196593 3285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:49.196635 3285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:48:49.199070 3285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:39307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.146.148:55517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:49.199347 3285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:49.199621 3285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:49.206557 3295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:49.206835 3296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:49.207033 3298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:49.208309 3285 server_base.cc:1034] running on GCE node
I20260812 14:48:49.208520 3285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:49.208808 3285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:49.209991 3285 hybrid_clock.cc:648] HybridClock initialized: now 1786546129209969 us; error 33 us; skew 500 ppm
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:49.213259 3285 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:48:49.213770 3285 webserver.cc:460] Webserver started at http://127.13.146.130:35327/ using document root <none> and password file <none>
I20260812 14:48:49.214021 3285 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:49.214076 3285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:49.214202 3285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:49.231083 3285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "fbf6d2f749b4432b8d1f64b6a9882b94"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "4be1f79d74b7399535b4b1efe9466118"
server_key_iv: "584d3841a14bbbfbbf7f54089c972e0d"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.231665 3285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "fbf6d2f749b4432b8d1f64b6a9882b94"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "4be1f79d74b7399535b4b1efe9466118"
server_key_iv: "584d3841a14bbbfbbf7f54089c972e0d"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.235080 3285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 14:48:49.253363 3315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.253639 3285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:48:49.253741 3285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "fbf6d2f749b4432b8d1f64b6a9882b94"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "4be1f79d74b7399535b4b1efe9466118"
server_key_iv: "584d3841a14bbbfbbf7f54089c972e0d"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.253840 3285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:49.268225 3285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:49.270968 3285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:49.271193 3285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:49.271495 3285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:49.271952 3285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:49.272045 3285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.272127 3285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:49.272190 3285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.280942 3285 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:40261
I20260812 14:48:49.281478 3285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:48:49.281836 3521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:40261 every 8 connection(s)
I20260812 14:48:49.288147 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 3285
I20260812 14:48:49.288292 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:48:49.288517 13898 external_mini_cluster.cc:1428] Setting key 61cbddb75e9d13bf1f9e9bc5c36c4b32
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:49.301440 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.289656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:46635 (local address 127.13.146.190:36333)
0812 14:48:49.289725 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.289727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.289738 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:48:49.290496 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.290497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.290514 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.290547 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:49.291517 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.292315 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.293921 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.294218 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.294311 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.297113 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:49.297114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:49.297115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:49.297124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:49.299333 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.300038 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.300040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.300041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.300111 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.300450 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.300451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.300451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.300609 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:49.300656 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.301112 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.301304 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:48:49.302515 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.289313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36333 (local address 127.13.146.130:46635)
0812 14:48:49.289560 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.289618 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.290359 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.290763 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.290767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.291070 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:49.291396 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.291404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.292451 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.292452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.293805 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.293808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.294389 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.294390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.294544 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.295197 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:49.295208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:49.296992 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:49.299458 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.299468 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.299471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.299938 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.300193 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.300193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.300194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.300365 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.300719 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:49.300726 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:49.300915 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.301847 (+ 932us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.302054 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
I20260812 14:48:49.303259 3522 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36333
I20260812 14:48:49.303369 3522 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:49.303584 3522 heartbeater.cc:511] Master 127.13.146.190:36333 requested a full tablet report, sending...
I20260812 14:48:49.303997 2896 ts_manager.cc:194] Registered new tserver with Master: fbf6d2f749b4432b8d1f64b6a9882b94 (127.13.146.130:40261)
I20260812 14:48:49.304507 2896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:46635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:48:49.335938 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:55517/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:39307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:48:49.479633 3530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:48:49.479944 3530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:48:49.480003 3530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:48:49.481985 3530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:48:49.482208 3530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:48:49.482299 3530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:48:49.484694 3530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:39307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.146.148:55517/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:48:49.485088 3530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:48:49.485416 3530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:48:49.492815 3544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:49.493276 3548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:48:49.493723 3545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:48:49.494442 3530 server_base.cc:1034] running on GCE node
I20260812 14:48:49.494740 3530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:48:49.495035 3530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:48:49.496208 3530 hybrid_clock.cc:648] HybridClock initialized: now 1786546129496184 us; error 37 us; skew 500 ppm
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:48:49.505754 3530 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:48:49.506811 3530 webserver.cc:460] Webserver started at http://127.13.146.131:41147/ using document root <none> and password file <none>
I20260812 14:48:49.507174 3530 fs_manager.cc:359] Metadata directory not provided
I20260812 14:48:49.507294 3530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:48:49.507450 3530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:48:49.527765 3530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "758bcc8c959747cd86ce87b837ed5181"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "08f9478aa3e6fd5d5cfe03d66ffdfbb5"
server_key_iv: "fe97f8b757701ce75451e00bc8a70647"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.528312 3530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "758bcc8c959747cd86ce87b837ed5181"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "08f9478aa3e6fd5d5cfe03d66ffdfbb5"
server_key_iv: "fe97f8b757701ce75451e00bc8a70647"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.530471 3530 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 14:48:49.564462 3562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.573378 3530 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.000s sys 0.000s
I20260812 14:48:49.573565 3530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "758bcc8c959747cd86ce87b837ed5181"
format_stamp: "Formatted at 2026-08-12 14:48:49 on dist-test-slave-8hhm"
server_key: "08f9478aa3e6fd5d5cfe03d66ffdfbb5"
server_key_iv: "fe97f8b757701ce75451e00bc8a70647"
server_key_version: "kuduclusterkey@0"
I20260812 14:48:49.573698 3530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:48:49.593116 3530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:48:49.595649 3530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:48:49.595813 3530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:48:49.596050 3530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:48:49.596448 3530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:48:49.596518 3530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.596565 3530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:48:49.596592 3530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:48:49.604764 3530 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36349
I20260812 14:48:49.607465 3775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36349 every 8 connection(s)
I20260812 14:48:49.607609 3530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:48:49.609081 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 3530
I20260812 14:48:49.609203 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:48:49.609499 13898 external_mini_cluster.cc:1428] Setting key 22d36da089ccd77776d429fc45d7d19f
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:48:49.620672 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.608622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:50163 (local address 127.13.146.190:36333)
0812 14:48:49.608670 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.608672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.609086 (+ 414us) server_negotiation.cc:407] Connection header received
0812 14:48:49.610591 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.610592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.610604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.610635 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:49.611561 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.612343 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.613919 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.614197 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.614256 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.617088 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:49.617090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:49.617090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:49.617099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:49.618958 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.619472 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.619473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.619473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.619553 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.619865 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.619866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.619866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.620005 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:49.620061 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.620470 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.620573 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:48:49.622044 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.608632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36333 (local address 127.13.146.131:50163)
0812 14:48:49.608935 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.608986 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.610461 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.610834 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.610838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.611133 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:48:49.611430 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.611438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.612493 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.612494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.613813 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.613817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.614361 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.614362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.614530 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.615180 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:48:49.615190 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:48:49.616969 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:48:49.619063 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.619070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.619073 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.619375 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.619633 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:48:49.619634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:48:49.619634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:48:49.619786 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:48:49.620115 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:48:49.620122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:48:49.620297 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.621334 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.621795 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":101,"threads_started":1}
I20260812 14:48:49.622730 3776 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36333
I20260812 14:48:49.622830 3776 heartbeater.cc:464] Registering TS with master...
I20260812 14:48:49.623027 3776 heartbeater.cc:511] Master 127.13.146.190:36333 requested a full tablet report, sending...
I20260812 14:48:49.623454 2896 ts_manager.cc:194] Registered new tserver with Master: 758bcc8c959747cd86ce87b837ed5181 (127.13.146.131:36349)
I20260812 14:48:49.623957 2896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:50163
I20260812 14:48:49.632426 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:48:49.644032 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.635414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.13.146.190:36333)
0812 14:48:49.635457 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.635459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.635879 (+ 420us) server_negotiation.cc:407] Connection header received
0812 14:48:49.635982 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.635983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.635994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.636025 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:48:49.636933 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.637728 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.638841 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.639130 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.639178 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.640877 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:48:49.640878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:48:49.640879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:48:49.640886 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:48:49.642726 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.643134 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.643135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.643136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.643198 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:48:49.643398 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:48:49.643399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:48:49.643400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:48:49.643531 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:48:49.643611 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.643852 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.643935 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:48:49.646467 2896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:48:49.648973 2896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:48:49.659194 3797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.652978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46175 (local address 127.0.0.1:54836)
0812 14:48:49.654132 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.654142 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.654192 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.654453 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.654456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.654458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:49.654535 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.654536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.656119 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.656120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.657857 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.657860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.658872 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.658873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.658957 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.658972 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.659048 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.659105 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":79,"threads_started":1}
I20260812 14:48:49.659658 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.653245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.13.146.129:46175)
0812 14:48:49.654079 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.654084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.654243 (+ 159us) server_negotiation.cc:407] Connection header received
0812 14:48:49.654282 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.654284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.654301 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.654370 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:49.654611 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.656018 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.657940 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.658686 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.658767 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.658832 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.659536 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.659582 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":62,"threads_started":1}
I20260812 14:48:49.660717 3138 tablet_service.cc:1467] Processing CreateTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 (DEFAULT_TABLE table=test-table [id=c7b276a3ebab48d8b37fb34074500322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:49.661128 3138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08d7eafeb554c9e8a363f223b79cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:49.674405 3807 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Bootstrap starting.
I20260812 14:48:49.675199 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.652427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40261 (local address 127.0.0.1:59408)
0812 14:48:49.653710 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.653753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.653820 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.667622 (+ 13802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.667623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.667626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:49.667701 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.667703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.669376 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.669377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.670521 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.670524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.674949 (+ 4425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.674950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.675027 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.675033 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.675068 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.675116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":90,"threads_started":1}
I20260812 14:48:49.675951 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.652538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.13.146.130:40261)
0812 14:48:49.665573 (+ 13035us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.665578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.665590 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:48:49.665635 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.665637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.665656 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.667783 (+ 2127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:49.667873 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.669267 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.670661 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.671435 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.675676 (+ 4241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.675776 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.675810 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.675861 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12988,"thread_start_us":80,"threads_started":1}
I20260812 14:48:49.676465 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.652664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36349 (local address 127.0.0.1:57700)
0812 14:48:49.653929 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.653938 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.653976 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.666092 (+ 12116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.666093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.666096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:49.666169 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.666171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.671574 (+ 5403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.671575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.672706 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.672709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.676228 (+ 3519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.676229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.676304 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.676309 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.676344 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.676390 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":73,"threads_started":1}
I20260812 14:48:49.677695 3807 tablet_bootstrap.cc:654] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:49.678911 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.652903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.13.146.131:36349)
0812 14:48:49.665902 (+ 12999us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.665907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.665918 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:48:49.665958 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.665960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.665976 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.666256 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:49.666340 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.667492 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.673178 (+ 5686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.674156 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.677890 (+ 3734us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.678684 (+ 794us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.678718 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.678781 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12944,"thread_start_us":94,"threads_started":1}
I20260812 14:48:49.679175 3404 tablet_service.cc:1467] Processing CreateTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 (DEFAULT_TABLE table=test-table [id=c7b276a3ebab48d8b37fb34074500322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:49.679603 3404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08d7eafeb554c9e8a363f223b79cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:49.679917 3655 tablet_service.cc:1467] Processing CreateTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 (DEFAULT_TABLE table=test-table [id=c7b276a3ebab48d8b37fb34074500322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:48:49.680258 3655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08d7eafeb554c9e8a363f223b79cb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:48:49.684896 3807 log.cc:824] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:49.685585 3810 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Bootstrap starting.
I20260812 14:48:49.686745 3810 tablet_bootstrap.cc:654] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:49.687150 3810 log.cc:824] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:49.688052 3807 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: No bootstrap required, opened a new log
I20260812 14:48:49.688182 3807 ts_tablet_manager.cc:1397] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260812 14:48:49.691133 3807 raft_consensus.cc:348] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.691466 3807 raft_consensus.cc:374] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:49.691572 3807 raft_consensus.cc:729] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d2ba773b334c61b1d493141cfdcc3f, State: Initialized, Role: FOLLOWER
I20260812 14:48:49.691843 3807 consensus_queue.cc:260] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.693029 3807 ts_tablet_manager.cc:1428] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 14:48:49.693240 3278 heartbeater.cc:503] Master 127.13.146.190:36333 was elected leader, sending a full tablet report...
I20260812 14:48:49.693421 3808 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Bootstrap starting.
I20260812 14:48:49.694525 3808 tablet_bootstrap.cc:654] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Neither blocks nor log segments found. Creating new log.
I20260812 14:48:49.694839 3808 log.cc:824] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Log is configured to *not* fsync() on all Append() calls
I20260812 14:48:49.695060 3810 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: No bootstrap required, opened a new log
I20260812 14:48:49.695139 3810 ts_tablet_manager.cc:1397] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260812 14:48:49.697500 3810 raft_consensus.cc:348] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.697777 3810 raft_consensus.cc:374] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:49.697872 3810 raft_consensus.cc:729] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758bcc8c959747cd86ce87b837ed5181, State: Initialized, Role: FOLLOWER
I20260812 14:48:49.698107 3810 consensus_queue.cc:260] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.698562 3810 ts_tablet_manager.cc:1428] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:48:49.698757 3776 heartbeater.cc:503] Master 127.13.146.190:36333 was elected leader, sending a full tablet report...
I20260812 14:48:49.700556 3808 tablet_bootstrap.cc:492] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: No bootstrap required, opened a new log
I20260812 14:48:49.700649 3808 ts_tablet_manager.cc:1397] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 14:48:49.703024 3808 raft_consensus.cc:348] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.703274 3808 raft_consensus.cc:374] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:48:49.703320 3808 raft_consensus.cc:729] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf6d2f749b4432b8d1f64b6a9882b94, State: Initialized, Role: FOLLOWER
I20260812 14:48:49.703460 3808 consensus_queue.cc:260] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.703754 3808 ts_tablet_manager.cc:1428] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:48:49.703896 3522 heartbeater.cc:503] Master 127.13.146.190:36333 was elected leader, sending a full tablet report...
Aug 12 14:48:49 dist-test-slave-8hhm krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546129, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
W20260812 14:48:49.776739 3280 tablet.cc:2369] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:48:49.789366 3523 tablet.cc:2369] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:48:49.858269 3777 tablet.cc:2369] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:48:49.880218 3820 raft_consensus.cc:482] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:48:49.880410 3820 raft_consensus.cc:504] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.881002 3820 leader_election.cc:288] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbf6d2f749b4432b8d1f64b6a9882b94 (127.13.146.130:40261), c8d2ba773b334c61b1d493141cfdcc3f (127.13.146.129:46175)
I20260812 14:48:49.884784 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.881211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40261 (local address 127.13.146.131:47187)
0812 14:48:49.881289 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.881301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.881351 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.881527 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.881528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.881530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:49.881602 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.881604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.882576 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.882577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.883757 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.883760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.884540 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.884541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.884618 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.884621 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.884666 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.884701 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 14:48:49.885077 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.881300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47187 (local address 127.13.146.130:40261)
0812 14:48:49.881343 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.881345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.881355 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:48:49.881418 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.881418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.881428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.881472 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:49.881688 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.882472 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.883874 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.884441 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.884502 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.884510 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.884960 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.885003 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:48:49.885390 3443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08d7eafeb554c9e8a363f223b79cb91" candidate_uuid: "758bcc8c959747cd86ce87b837ed5181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" is_pre_election: true
I20260812 14:48:49.885571 3443 raft_consensus.cc:2454] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758bcc8c959747cd86ce87b837ed5181 in term 0.
I20260812 14:48:49.885847 3567 leader_election.cc:302] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758bcc8c959747cd86ce87b837ed5181, fbf6d2f749b4432b8d1f64b6a9882b94; no voters:
I20260812 14:48:49.886098 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.881794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:56911 (local address 127.13.146.129:46175)
0812 14:48:49.881858 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.881861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.881927 (+ 66us) server_negotiation.cc:407] Connection header received
0812 14:48:49.882021 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.882022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.882032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.882061 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:48:49.882266 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.882933 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.884258 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.884829 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.884876 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.884884 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.885988 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.886032 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:48:49.886081 3820 raft_consensus.cc:2781] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:48:49.886142 3820 raft_consensus.cc:482] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:48:49.886173 3820 raft_consensus.cc:3037] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:49.886368 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.881600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46175 (local address 127.13.146.131:56911)
0812 14:48:49.881867 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:49.881878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:48:49.881951 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:48:49.882125 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:48:49.882126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:48:49.882127 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:48:49.882200 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.882201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.883028 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.883029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.884167 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:48:49.884169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.885820 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.885821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:48:49.885894 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.885897 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:48:49.886202 (+ 305us) client_negotiation.cc:240] Negotiation successful
0812 14:48:49.886251 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":97,"threads_started":1}
I20260812 14:48:49.886775 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08d7eafeb554c9e8a363f223b79cb91" candidate_uuid: "758bcc8c959747cd86ce87b837ed5181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" is_pre_election: true
I20260812 14:48:49.886922 3200 raft_consensus.cc:2454] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758bcc8c959747cd86ce87b837ed5181 in term 0.
I20260812 14:48:49.888310 3820 raft_consensus.cc:504] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.888463 3820 leader_election.cc:288] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [CANDIDATE]: Term 1 election: Requested vote from peers fbf6d2f749b4432b8d1f64b6a9882b94 (127.13.146.130:40261), c8d2ba773b334c61b1d493141cfdcc3f (127.13.146.129:46175)
I20260812 14:48:49.888643 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08d7eafeb554c9e8a363f223b79cb91" candidate_uuid: "758bcc8c959747cd86ce87b837ed5181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8d2ba773b334c61b1d493141cfdcc3f"
I20260812 14:48:49.888731 3200 raft_consensus.cc:3037] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:49.889551 3200 raft_consensus.cc:2454] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758bcc8c959747cd86ce87b837ed5181 in term 1.
I20260812 14:48:49.889683 3443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08d7eafeb554c9e8a363f223b79cb91" candidate_uuid: "758bcc8c959747cd86ce87b837ed5181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94"
I20260812 14:48:49.889786 3443 raft_consensus.cc:3037] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:48:49.890478 3443 raft_consensus.cc:2454] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758bcc8c959747cd86ce87b837ed5181 in term 1.
I20260812 14:48:49.890652 3567 leader_election.cc:302] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758bcc8c959747cd86ce87b837ed5181, fbf6d2f749b4432b8d1f64b6a9882b94; no voters:
I20260812 14:48:49.890784 3820 raft_consensus.cc:2781] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:48:49.890971 3820 raft_consensus.cc:686] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 1 LEADER]: Becoming Leader. State: Replica: 758bcc8c959747cd86ce87b837ed5181, State: Running, Role: LEADER
I20260812 14:48:49.891086 3820 consensus_queue.cc:237] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } }
I20260812 14:48:49.891877 2896 catalog_manager.cc:5495] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 reported cstate change: term changed from 0 to 1, leader changed from <none> to 758bcc8c959747cd86ce87b837ed5181 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "758bcc8c959747cd86ce87b837ed5181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "758bcc8c959747cd86ce87b837ed5181" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 } health_report { overall_health: UNKNOWN } } }
I20260812 14:48:49.943228 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:49.940606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.13.146.131:36349)
0812 14:48:49.940725 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 14:48:49.940727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:48:49.940733 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:48:49.940767 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:48:49.940768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:48:49.940784 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:48:49.940865 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:48:49.941032 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.941656 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.942421 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:48:49.942635 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:48:49.942711 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:48:49.942884 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:48:49.942999 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:48:49.943099 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:48:49.943118 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 14:48:49.943140 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 14:48:49.945165 3443 raft_consensus.cc:1261] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 1 FOLLOWER]: Refusing update from remote peer 758bcc8c959747cd86ce87b837ed5181: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:48:49.945178 3200 raft_consensus.cc:1261] T a08d7eafeb554c9e8a363f223b79cb91 P c8d2ba773b334c61b1d493141cfdcc3f [term 1 FOLLOWER]: Refusing update from remote peer 758bcc8c959747cd86ce87b837ed5181: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:48:49.945554 3820 consensus_queue.cc:1035] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbf6d2f749b4432b8d1f64b6a9882b94" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:48:49.945736 3820 consensus_queue.cc:1035] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8d2ba773b334c61b1d493141cfdcc3f" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:48:49.951231 3827 mvcc.cc:205] Tried to move back new op lower bound from 7317692948252467200 to 7317692948034330624. Current Snapshot: MvccSnapshot[applied={T|T < 7317692948252467200}]
I20260812 14:48:49.951709 3831 mvcc.cc:205] Tried to move back new op lower bound from 7317692948252467200 to 7317692948034330624. Current Snapshot: MvccSnapshot[applied={T|T < 7317692948252467200}]
I20260812 14:48:49.955013 2895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:48:49.955127 2895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:48:49.955991 2895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a11345db35da4232ab1ce689eafe60b1: Sending DeleteTablet for 3 replicas of tablet a08d7eafeb554c9e8a363f223b79cb91
I20260812 14:48:49.956398 3404 tablet_service.cc:1514] Processing DeleteTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:59408
I20260812 14:48:49.956825 3138 tablet_service.cc:1514] Processing DeleteTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:54836
I20260812 14:48:49.956990 3655 tablet_service.cc:1514] Processing DeleteTablet for tablet a08d7eafeb554c9e8a363f223b79cb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:48:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57700
I20260812 14:48:49.957695 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 3052
I20260812 14:48:49.969334 3841 tablet_replica.cc:330] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: stopping tablet replica
I20260812 14:48:49.969491 3841 raft_consensus.cc:2229] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:48:49.969630 3841 raft_consensus.cc:2258] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:48:49.969811 3845 tablet_replica.cc:330] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: stopping tablet replica
I20260812 14:48:49.969916 3845 raft_consensus.cc:2229] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:48:49.970046 3845 raft_consensus.cc:2258] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:48:49.970445 3841 ts_tablet_manager.cc:1905] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:48:49.970515 3845 ts_tablet_manager.cc:1905] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:48:49.972349 3845 ts_tablet_manager.cc:1918] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:48:49.972545 3841 ts_tablet_manager.cc:1918] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:48:49.972687 3841 log.cc:1196] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/a08d7eafeb554c9e8a363f223b79cb91
I20260812 14:48:49.973044 3841 ts_tablet_manager.cc:1939] T a08d7eafeb554c9e8a363f223b79cb91 P fbf6d2f749b4432b8d1f64b6a9882b94: Deleting consensus metadata
I20260812 14:48:49.973627 2866 catalog_manager.cc:4841] TS fbf6d2f749b4432b8d1f64b6a9882b94 (127.13.146.130:40261): tablet a08d7eafeb554c9e8a363f223b79cb91 (table test-table [id=c7b276a3ebab48d8b37fb34074500322]) successfully deleted
I20260812 14:48:49.973860 3845 log.cc:1196] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/a08d7eafeb554c9e8a363f223b79cb91
I20260812 14:48:49.974227 3845 ts_tablet_manager.cc:1939] T a08d7eafeb554c9e8a363f223b79cb91 P 758bcc8c959747cd86ce87b837ed5181: Deleting consensus metadata
I20260812 14:48:49.974707 2866 catalog_manager.cc:4841] TS 758bcc8c959747cd86ce87b837ed5181 (127.13.146.131:36349): tablet a08d7eafeb554c9e8a363f223b79cb91 (table test-table [id=c7b276a3ebab48d8b37fb34074500322]) successfully deleted
W20260812 14:48:49.977924 2867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:46175 (error 108)
I20260812 14:48:49.978816 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 3285
I20260812 14:48:50.017786 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:50.017594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46175 (local address 127.0.0.1:54838)
0812 14:48:50.017668 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:50.017733 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 14:48:50.027403 2867 catalog_manager.cc:4584] TS c8d2ba773b334c61b1d493141cfdcc3f (127.13.146.129:46175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a08d7eafeb554c9e8a363f223b79cb91: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46175: connect: Connection refused (error 111)
I20260812 14:48:50.053542 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 3530
I20260812 14:48:50.080451 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:50.080301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46175 (local address 127.0.0.1:54854)
0812 14:48:50.080364 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:50.080397 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:48:50.080804 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:48:50.080720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:46175 (local address 127.0.0.1:54858)
0812 14:48:50.080752 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 14:48:50.080773 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 14:48:50.082346 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 2835
I20260812 14:48:50.094233 13898 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:48:50.441874 13898 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:48:50.441969 13898 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:48:50.787786 13898 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:48:50.788 UTC [2058] LOG: received smart shutdown request
2026-08-12 14:48:50.792 UTC [2058] LOG: background worker "logical replication launcher" (PID 2077) exited with exit code 1
2026-08-12 14:48:50.792 UTC [2067] LOG: shutting down
2026-08-12 14:48:50.968 UTC [2058] LOG: database system is shut down
2026-08-12T14:48:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48522 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:48:51 dist-test-slave-8hhm krb5kdc[3869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:48:51 dist-test-slave-8hhm krb5kdc[3869](info): set up 2 sockets
Aug 12 14:48:51 dist-test-slave-8hhm krb5kdc[3869](info): commencing operation
krb5kdc: starting...
W20260812 14:48:52.391098 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:48:52 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:48:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangeradmin_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangeradmin_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangerlookup_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangerlookup_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
I20260812 14:48:52.567593 13898 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskS5QCDN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/postgres -l logfile start
2026-08-12 14:48:55.116 UTC [3935] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:48:55.116 UTC [3935] LOG: listening on IPv4 address "127.13.146.148", port 33455
2026-08-12 14:48:55.119 UTC [3935] LOG: listening on Unix socket "/tmp/.s.PGSQL.33455"
2026-08-12 14:48:55.141 UTC [3940] LOG: database system was shut down at 2026-08-12 14:48:53 UTC
2026-08-12 14:48:55.145 UTC [3935] LOG: database system is ready to accept connections
I20260812 14:48:56.357355 13898 mini_postgres.cc:96] Postgres bound to 33455
2026-08-12 14:48:56.364 UTC [3950] FATAL: database "slave" does not exist
127.13.146.148:33455 - accepting connections
I20260812 14:48:56.365177 13898 mini_ranger.cc:161] Starting Ranger...
I20260812 14:48:56.381286 13898 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:48:56.860788 13898 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:48:56.860877 13898 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin
2026-08-12 14:48:57,156 [I] DB FLAVOR :POSTGRES
2026-08-12 14:48:57,156 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:48:57,157 [I] Checking connection..
2026-08-12 14:48:57,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:48:57,436 [I] Checking connection passed.
2026-08-12 14:48:57,436 [I] --------- Verifying version history table ---------
2026-08-12 14:48:57,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:57,709 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:48:57,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:57,974 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:48:57,974 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:48:57,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:48:58.234 UTC [4056] WARNING: there is no transaction in progress
2026-08-12 14:48:58,244 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:48:58,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:48:58,509 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:48:58,509 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:48:58,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:48:58,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:48:59,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-12 14:48:59,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:48:59,568 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:48:59,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:48:59,844 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:48:59,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:49:00,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:49:00,372 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:49:00,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:49:01.356 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.373 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.389 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.405 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.421 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.439 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.510 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.517 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.533 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.544 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.554 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.569 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.579 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.585 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.592 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:01.599 UTC [4245] WARNING: there is no transaction in progress
2026-08-12 14:49:02,262 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:49:02,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:49:02,529 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:49:02,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:49:02,812 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:49:02,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:49:03,078 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:49:03,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:49:03,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:49:03,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-12 14:49:03,840 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:49:03,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:49:04,093 [I] DB_PATCHES have already been applied
I20260812 14:49:04.099383 13898 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskS5QCDN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:49:04.099469 13898 mini_ranger.cc:193] Using host: 127.13.146.148
I20260812 14:49:04.103626 13898 mini_ranger.cc:227] Ranger admin URL: http://127.13.146.148:55347
Aug 12, 2026 2:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:49:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:49:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:49:05.548529 13898 mini_ranger.cc:160] Time spent starting Ranger: real 9.183s user 0.004s sys 0.007s
Aug 12, 2026 2:49:25 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:49:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 2:49:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:49:25 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:49:26 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:49:26.775137 13898 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.146.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.146.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangerkms_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.146.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/rangerkms_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.146.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.146.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.146.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/HTTP_127.13.146.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:49:26 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546166, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:49:26.871153 13898 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:49:26.881776 13898 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:49:27.852418 13898 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:49:27.852517 13898 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms
I20260812 14:49:27.852538 13898 mini_ranger_kms.cc:227] Using postgres at 127.13.146.148:33455
2026-08-12 14:49:28,022 [I] DB FLAVOR :POSTGRES
2026-08-12 14:49:28,022 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:49:28,023 [I] Checking connection
2026-08-12 14:49:28,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:49:28,280 [I] connection success
2026-08-12 14:49:28,281 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:49:28,281 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:49:28,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:49:28,533 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:49:28,534 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:49:28,534 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:49:28,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.146.148:33455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:49:28,828 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:49:29.385932 13898 mini_ranger_kms.cc:310] Created kms service
I20260812 14:49:29.571727 13898 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:49:29.620411 13898 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:49:29 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546166, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:30.063896 13898 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:49:30.064122 13898 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms:/tmp/dist-test-taskS5QCDN/build/release/bin/postgresql.jar:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskS5QCDN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:49:30.064150 13898 mini_ranger_kms.cc:240] Using host: 127.13.146.148
I20260812 14:49:30.068604 13898 mini_ranger_kms.cc:276] Ranger KMS PID: 4603
I20260812 14:49:30.068711 13898 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.146.148:47713
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:49:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 14:49:32 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786546172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:33.088166 13898 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.217s user 0.000s sys 0.153s
I20260812 14:49:33.088297 13898 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:49:33.088358 13898 mini_ranger_kms.cc:401] 127.13.146.148:47713/kms/v1/keys
Aug 12 14:49:33 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
Aug 12 14:49:33 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.146.148@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
Aug 12 14:49:36 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:49:36.208189 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44987
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:44987
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:36.293442 4700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:36.293672 4700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:36.293695 4700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:36.294931 4700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:36.294978 4700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:36.294992 4700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:36.295004 4700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:36.295017 4700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:36.296653 4700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.146.148:47713/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:44987
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:36.296911 4700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:36.297174 4700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:36.302263 4709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:36.302263 4714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:36.302304 4710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:36.302920 4700 server_base.cc:1034] running on GCE node
I20260812 14:49:36.303263 4700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:36.303566 4700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:36.304742 4700 hybrid_clock.cc:648] HybridClock initialized: now 1786546176304738 us; error 41 us; skew 500 ppm
Aug 12 14:49:36 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546176, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:36.310536 4700 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:36.311015 4700 webserver.cc:460] Webserver started at http://127.13.146.190:34155/ using document root <none> and password file <none>
I20260812 14:49:36.311277 4700 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:36.311321 4700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:36.311458 4700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:49:36 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546176, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:36.375277 4700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "a3c73991d9e349fca084135fa1e8145e"
format_stamp: "Formatted at 2026-08-12 14:49:36 on dist-test-slave-8hhm"
server_key: "160d2e346d9f12d44b2bb2691dd30751"
server_key_iv: "0f5ebc3700a6d8bf3bdeb416b07f2fe7"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:36.375711 4700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "a3c73991d9e349fca084135fa1e8145e"
format_stamp: "Formatted at 2026-08-12 14:49:36 on dist-test-slave-8hhm"
server_key: "160d2e346d9f12d44b2bb2691dd30751"
server_key_iv: "0f5ebc3700a6d8bf3bdeb416b07f2fe7"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:36.377386 4700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 14:49:36.392012 4729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:36.392340 4700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:36.392484 4700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a3c73991d9e349fca084135fa1e8145e"
format_stamp: "Formatted at 2026-08-12 14:49:36 on dist-test-slave-8hhm"
server_key: "160d2e346d9f12d44b2bb2691dd30751"
server_key_iv: "0f5ebc3700a6d8bf3bdeb416b07f2fe7"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:36.392620 4700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:36.407440 4700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:36.409679 4700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:36.409868 4700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:36.414999 4700 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:44987
I20260812 14:49:36.415051 4827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:44987 every 8 connection(s)
I20260812 14:49:36.415479 4700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:36.416257 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:36.418253 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 4700
I20260812 14:49:36.418368 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:49:36.418555 13898 external_mini_cluster.cc:1428] Setting key 3c27041e47b538fe6101984337f92d7b
I20260812 14:49:36.419070 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Bootstrap starting.
I20260812 14:49:36.419793 4830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:36.420185 4830 log.cc:824] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:36.421061 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: No bootstrap required, opened a new log
I20260812 14:49:36.423002 4830 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:36.423202 4830 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:36.423234 4830 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c73991d9e349fca084135fa1e8145e, State: Initialized, Role: FOLLOWER
I20260812 14:49:36.423360 4830 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:36.423432 4830 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:36.423498 4830 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:36.423547 4830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:36.424292 4830 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:36.424417 4830 leader_election.cc:302] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c73991d9e349fca084135fa1e8145e; no voters:
I20260812 14:49:36.424664 4830 leader_election.cc:288] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:49:36.424716 4838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:36.424994 4838 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 LEADER]: Becoming Leader. State: Replica: a3c73991d9e349fca084135fa1e8145e, State: Running, Role: LEADER
I20260812 14:49:36.425058 4830 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:36.425154 4838 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:36.425657 4838 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c73991d9e349fca084135fa1e8145e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } } }
I20260812 14:49:36.425818 4838 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:36.426010 4838 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c73991d9e349fca084135fa1e8145e. Latest consensus state: current_term: 1 leader_uuid: "a3c73991d9e349fca084135fa1e8145e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } } }
I20260812 14:49:36.426132 4838 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:36.426306 4849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:36.426657 4830 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:49:36.426887 4849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:49:36 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:36.434628 4849 catalog_manager.cc:1347] Generated new cluster ID: d8d5cccba470437293b34d81a900ac25
I20260812 14:49:36.434787 4849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:49:36.435477 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:36.419129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48240 (local address 127.13.146.190:44987)
0812 14:49:36.420486 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:36.420497 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:36.420532 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:49:36.420998 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:36.421006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:36.421067 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:36.421257 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:36.421998 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:36.423260 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:36.424197 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:36.424809 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:36.429146 (+ 4337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:36.429761 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:36.429771 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:36.429771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:36.429783 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:36.433044 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:36.433575 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:36.433578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:36.433579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:36.433698 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:36.433923 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:36.433924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:36.433924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:36.434314 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:36.434950 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:36.435035 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:49:36.435207 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310,"thread_start_us":81,"threads_started":1}
I20260812 14:49:36.452270 4849 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:49:36.453040 4849 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:36.462309 4849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Generated new TSK 0
I20260812 14:49:36.462528 4849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:49:37.173660 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:37.264834 4922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:37.265095 4922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:37.265136 4922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:37.266494 4922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:37.266579 4922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:37.266635 4922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:49:37.268213 4922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:37.268553 4922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:37.268860 4922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:37.275507 4929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.275507 4932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.275856 4928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:37.275950 4922 server_base.cc:1034] running on GCE node
I20260812 14:49:37.276317 4922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:37.276667 4922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:37.277861 4922 hybrid_clock.cc:648] HybridClock initialized: now 1786546177277843 us; error 28 us; skew 500 ppm
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:37.281266 4922 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:49:37.281821 4922 webserver.cc:460] Webserver started at http://127.13.146.129:45171/ using document root <none> and password file <none>
I20260812 14:49:37.282178 4922 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:37.282260 4922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:37.282405 4922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:37.298121 4922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "b16ae72595ca4b0c9c04600a4b6198d6"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "9eaf67b2401cdb65177bfea8ded6885c"
server_key_iv: "645c8aca58d0c30437948a085bf8cc97"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.298630 4922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "b16ae72595ca4b0c9c04600a4b6198d6"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "9eaf67b2401cdb65177bfea8ded6885c"
server_key_iv: "645c8aca58d0c30437948a085bf8cc97"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.300664 4922 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 14:49:37.315420 4945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.315733 4922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 14:49:37.315848 4922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "b16ae72595ca4b0c9c04600a4b6198d6"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "9eaf67b2401cdb65177bfea8ded6885c"
server_key_iv: "645c8aca58d0c30437948a085bf8cc97"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.315951 4922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:37.329372 4922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:37.332103 4922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:37.332372 4922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:37.332702 4922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:37.333174 4922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:37.333294 4922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.333410 4922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:37.333492 4922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.340266 4922 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:33099
I20260812 14:49:37.340834 4922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:49:37.341434 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 4922
I20260812 14:49:37.341554 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:49:37.341754 13898 external_mini_cluster.cc:1428] Setting key b4854d986a36f14f3d51d482f4fca276
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
I20260812 14:49:37.352947 5152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:33099 every 8 connection(s)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:37.367568 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.353937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:56501 (local address 127.13.146.190:44987)
0812 14:49:37.354700 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:37.354702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:37.354712 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:49:37.355108 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:37.355110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:37.355126 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:37.355197 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:37.357879 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.358697 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.360290 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.360633 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.360723 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.363674 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:37.363676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:37.363676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:37.363684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:37.365791 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.366370 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.366372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.366372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.366452 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.366711 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.366712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.366713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.366881 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:37.366965 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:37.367346 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 14:49:37.367452 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":104,"threads_started":1}
I20260812 14:49:37.367650 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.353927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.129:56501)
0812 14:49:37.354123 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:37.354168 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:37.354926 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:37.355394 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:37.355400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:37.355692 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:37.357743 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.357752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.358823 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.358824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.360172 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.360176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.360760 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.360761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.360933 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.361642 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:37.361655 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:37.363542 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:37.365901 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.365910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.365914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.366263 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.366528 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.366529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.366529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.366639 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.367030 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:37.367038 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:37.367177 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 14:49:37.367267 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:49:37.367397 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260812 14:49:37.368520 5153 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:37.368641 5153 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:37.368851 5153 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:37.369560 4760 ts_manager.cc:194] Registered new tserver with Master: b16ae72595ca4b0c9c04600a4b6198d6 (127.13.146.129:33099)
I20260812 14:49:37.370563 4760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:56501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:49:37.377900 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:37.490999 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:37.491276 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:37.491321 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:37.492574 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:37.492654 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:37.492702 5166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:49:37.494364 5166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:37.494660 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:37.494943 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:37.500560 5188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.500517 5180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.500916 5183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:37.501549 5166 server_base.cc:1034] running on GCE node
I20260812 14:49:37.501767 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:37.502068 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:37.505306 5166 hybrid_clock.cc:648] HybridClock initialized: now 1786546177505291 us; error 28 us; skew 500 ppm
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:37.508201 5166 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:49:37.508857 5166 webserver.cc:460] Webserver started at http://127.13.146.130:41423/ using document root <none> and password file <none>
I20260812 14:49:37.509117 5166 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:37.509191 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:37.509389 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:37.528677 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "157e257ac5516f24e6976825321979e0"
server_key_iv: "5b6991b91000205851665aec895f879e"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.529147 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "157e257ac5516f24e6976825321979e0"
server_key_iv: "5b6991b91000205851665aec895f879e"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.533634 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260812 14:49:37.552990 5198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.553284 5166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:49:37.553395 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "157e257ac5516f24e6976825321979e0"
server_key_iv: "5b6991b91000205851665aec895f879e"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.553496 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:37.578035 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:37.580670 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:37.580894 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:37.581179 5166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:37.582758 5166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:37.582857 5166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.582923 5166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:37.582970 5166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.591931 5166 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:39327
I20260812 14:49:37.592485 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:49:37.593078 5417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:39327 every 8 connection(s)
I20260812 14:49:37.596246 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 5166
I20260812 14:49:37.596352 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:49:37.596549 13898 external_mini_cluster.cc:1428] Setting key 3f540f50ef7b450eccbd420f183353ca
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:37.606377 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.593921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57629 (local address 127.13.146.190:44987)
0812 14:49:37.593974 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:37.593977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:37.595127 (+ 1150us) server_negotiation.cc:407] Connection header received
0812 14:49:37.596139 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:37.596141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:37.596160 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:37.596192 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:37.597333 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.598252 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.599782 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.600092 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.600140 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.602904 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:37.602906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:37.602906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:37.602914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:37.604743 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.605254 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.605255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.605256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.605317 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.605624 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.605626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.605626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.605763 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:37.605816 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:37.606182 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 14:49:37.606280 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:49:37.607029 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.593906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.130:57629)
0812 14:49:37.594982 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:37.595031 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:37.595999 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:37.596590 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:37.596627 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:37.596888 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:37.597176 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.597183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.598373 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.598374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.599675 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.599678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.600252 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.600253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.600395 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.601061 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:37.601072 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:37.602791 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:37.604842 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.604848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.604850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.605142 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.605393 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.605394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.605394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.605529 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.605851 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:37.605857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:37.606019 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 14:49:37.606629 (+ 610us) client_negotiation.cc:240] Negotiation successful
0812 14:49:37.606792 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":95,"threads_started":1}
I20260812 14:49:37.607677 5418 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:37.607775 5418 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:37.607982 5418 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:49:37.608707 4760 ts_manager.cc:194] Registered new tserver with Master: 3bab7c2a6c6544a1aa22b2649a1c03aa (127.13.146.130:39327)
I20260812 14:49:37.609206 4760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:57629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:49:37.632417 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:37.760555 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:37.760874 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:37.760916 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:37.762684 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:37.762756 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:37.762796 5426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:49:37.764972 5426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:37.765295 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:37.765637 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:37.772154 5436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.772164 5433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:37.772163 5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:37.773115 5426 server_base.cc:1034] running on GCE node
I20260812 14:49:37.773455 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:37.773732 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:37.774922 5426 hybrid_clock.cc:648] HybridClock initialized: now 1786546177774897 us; error 37 us; skew 500 ppm
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:37.778659 5426 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:49:37.779320 5426 webserver.cc:460] Webserver started at http://127.13.146.131:41399/ using document root <none> and password file <none>
I20260812 14:49:37.779655 5426 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:37.779821 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:37.780025 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:37.796214 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "0baed1e7a9f345a5a34ffaee793b39fb"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "342785a909dad4c48fdfbd911bf86431"
server_key_iv: "3d0a06997ec8ebedd117f0fdfefcb532"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.796795 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "0baed1e7a9f345a5a34ffaee793b39fb"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "342785a909dad4c48fdfbd911bf86431"
server_key_iv: "3d0a06997ec8ebedd117f0fdfefcb532"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.798839 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 14:49:37.827239 5449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.827518 5426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 14:49:37.827637 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "0baed1e7a9f345a5a34ffaee793b39fb"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "342785a909dad4c48fdfbd911bf86431"
server_key_iv: "3d0a06997ec8ebedd117f0fdfefcb532"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:37.827735 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:37.844925 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:37.847472 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:37.847641 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:37.847923 5426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:37.848320 5426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:37.848372 5426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.848403 5426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:37.848421 5426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:37.856633 5426 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:35173
I20260812 14:49:37.856812 5664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:35173 every 8 connection(s)
I20260812 14:49:37.857125 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:49:37.861477 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 5426
I20260812 14:49:37.861598 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:49:37.861786 13898 external_mini_cluster.cc:1428] Setting key 1e0daf8323f0feeea5f597bb31d24e1b
Aug 12 14:49:37 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546177, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:37.867504 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.857998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:41655 (local address 127.13.146.190:44987)
0812 14:49:37.858058 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:37.858060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:37.858291 (+ 231us) server_negotiation.cc:407] Connection header received
0812 14:49:37.858797 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:37.858798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:37.858812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:37.858896 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:37.859613 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.860388 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.861470 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.861741 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.861831 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.864522 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:37.864523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:37.864524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:37.864531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:37.866057 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.866489 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.866491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.866491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.866555 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:37.866827 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:37.866840 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:37.866840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:37.866954 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:37.867030 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:37.867328 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 14:49:37.867413 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:49:37.867689 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:37.857942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.131:41655)
0812 14:49:37.858147 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:37.858192 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:37.858686 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:37.858975 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:37.858978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:37.859300 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:37.859522 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.859527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.860489 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.860490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.861376 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:37.861379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:37.861856 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:37.861857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:37.862002 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:37.862672 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:37.862682 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:37.864424 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:37.866164 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.866171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.866173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.866411 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.866625 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:37.866626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:37.866626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:37.866750 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:37.867021 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:37.867028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:37.867185 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 14:49:37.867304 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 14:49:37.867450 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260812 14:49:37.868321 5666 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:37.868415 5666 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:37.868602 5666 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:37.869053 4760 ts_manager.cc:194] Registered new tserver with Master: 0baed1e7a9f345a5a34ffaee793b39fb (127.13.146.131:35173)
I20260812 14:49:37.869472 4760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:41655
I20260812 14:49:37.873705 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:49:37.874395 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 4922
I20260812 14:49:37.882894 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 5166
I20260812 14:49:37.893616 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 5426
I20260812 14:49:37.910068 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 4700
I20260812 14:49:37.922521 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44987
--webserver_interface=127.13.146.190
--webserver_port=34155
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:44987
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:38.006543 5669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:38.006758 5669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:38.006783 5669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:38.007869 5669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:38.007910 5669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:38.007926 5669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:38.007941 5669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:38.007954 5669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:38.009853 5669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:44987
--ranger_config_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:44987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:38.010049 5669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:38.010231 5669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:38.015300 5678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.015305 5684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.015305 5688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:38.015594 5669 server_base.cc:1034] running on GCE node
I20260812 14:49:38.015832 5669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:38.016145 5669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:38.017295 5669 hybrid_clock.cc:648] HybridClock initialized: now 1786546178017271 us; error 39 us; skew 500 ppm
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:38.020097 5669 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:38.020687 5669 webserver.cc:460] Webserver started at http://127.13.146.190:34155/ using document root <none> and password file <none>
I20260812 14:49:38.020901 5669 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:38.020956 5669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:38.097353 5669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:49:38.098347 5702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.098592 5669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:38.098726 5669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a3c73991d9e349fca084135fa1e8145e"
format_stamp: "Formatted at 2026-08-12 14:49:36 on dist-test-slave-8hhm"
server_key: "160d2e346d9f12d44b2bb2691dd30751"
server_key_iv: "0f5ebc3700a6d8bf3bdeb416b07f2fe7"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:38.099212 5669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:38.133869 5669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:38.135905 5669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:38.136085 5669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:38.140882 5669 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:44987
I20260812 14:49:38.140923 5809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:44987 every 8 connection(s)
I20260812 14:49:38.141381 5669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:38.142762 5812 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:49:38.143385 5812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Bootstrap starting.
I20260812 14:49:38.150148 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 5669
I20260812 14:49:38.150506 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:33099
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:44987
--webserver_port=45171
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:38.151664 5812 log.cc:824] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:38.154376 5812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:49:38.154719 5812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Bootstrap complete.
I20260812 14:49:38.158980 5812 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:38.159422 5812 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c73991d9e349fca084135fa1e8145e, State: Initialized, Role: FOLLOWER
I20260812 14:49:38.159636 5812 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:38.159734 5812 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:38.159796 5812 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:38.159857 5812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:49:38.161635 5812 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:38.161773 5812 leader_election.cc:302] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c73991d9e349fca084135fa1e8145e; no voters:
I20260812 14:49:38.161967 5812 leader_election.cc:288] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:49:38.162043 5818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:49:38.162199 5818 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [term 2 LEADER]: Becoming Leader. State: Replica: a3c73991d9e349fca084135fa1e8145e, State: Running, Role: LEADER
I20260812 14:49:38.162288 5812 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:38.162292 5818 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } }
I20260812 14:49:38.162617 5818 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c73991d9e349fca084135fa1e8145e. Latest consensus state: current_term: 2 leader_uuid: "a3c73991d9e349fca084135fa1e8145e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } } }
I20260812 14:49:38.162686 5818 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:38.162824 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a3c73991d9e349fca084135fa1e8145e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c73991d9e349fca084135fa1e8145e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 44987 } } }
I20260812 14:49:38.162869 5826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:38.162892 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:38.163364 5826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:38.163540 5826 catalog_manager.cc:1259] Loaded cluster ID: d8d5cccba470437293b34d81a900ac25
I20260812 14:49:38.163594 5826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:49:38.164458 5812 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:49:38.164644 5826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:38.164836 5826 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Loaded TSK: 0
I20260812 14:49:38.165055 5826 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:49:38.248051 5817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:38.248284 5817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:38.248327 5817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:38.249781 5817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:38.249853 5817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:38.249898 5817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:49:38.251646 5817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:33099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=45171
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:38.251950 5817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:38.252171 5817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:38.258064 5881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.258064 5882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.258518 5887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:38.258852 5817 server_base.cc:1034] running on GCE node
I20260812 14:49:38.259066 5817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:38.259348 5817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:38.260527 5817 hybrid_clock.cc:648] HybridClock initialized: now 1786546178260516 us; error 31 us; skew 500 ppm
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:38.263389 5817 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:49:38.264074 5817 webserver.cc:460] Webserver started at http://127.13.146.129:45171/ using document root <none> and password file <none>
I20260812 14:49:38.264425 5817 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:38.264603 5817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:38.289589 5817 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:49:38.290746 5906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.290930 5817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:49:38.291036 5817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "b16ae72595ca4b0c9c04600a4b6198d6"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "9eaf67b2401cdb65177bfea8ded6885c"
server_key_iv: "645c8aca58d0c30437948a085bf8cc97"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:38.291469 5817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:38.310426 5817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:38.312744 5817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:38.312934 5817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:38.313239 5817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:38.313645 5817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:38.313706 5817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.313777 5817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:38.313822 5817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.320920 5817 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:33099
I20260812 14:49:38.321630 5817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:49:38.323061 6130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:33099 every 8 connection(s)
I20260812 14:49:38.328253 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 5817
I20260812 14:49:38.328583 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:39327
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:44987
--webserver_port=41423
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:38.337077 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.323919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.129:33003)
0812 14:49:38.324096 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:38.324140 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:38.324750 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:38.325371 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:38.325376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:38.325609 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:38.325799 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.325803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.327538 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.327539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.328762 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.328766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.329685 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.329687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.329835 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.330470 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:38.330483 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:38.332172 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:38.334798 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.334806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.334809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.335193 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.335537 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.335538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.335538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.335697 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.336261 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:38.336268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:38.336439 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 14:49:38.336549 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:49:38.336743 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260812 14:49:38.338513 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.324341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:33003 (local address 127.13.146.190:44987)
0812 14:49:38.324926 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:38.324934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:38.324963 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:49:38.325089 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:38.325093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:38.325136 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:38.325315 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:38.325927 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.327389 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.328887 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.329557 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.330133 (+ 576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.332313 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:38.332318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:38.332319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:38.332332 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:38.334691 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.335309 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.335312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.335313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.335434 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.335781 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.335782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.335782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.336159 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:38.337960 (+ 1801us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:38.338098 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 14:49:38.338246 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":86,"threads_started":1}
I20260812 14:49:38.339579 6133 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:38.339687 6133 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.339912 6133 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.340953 6133 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:44987 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:49:38.341049 6133 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.341128 6133 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:38.341631 6133 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.341717 6133 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.342145 6133 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:49:38.457912 6138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:38.458256 6138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:38.458307 6138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:38.460237 6138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:38.460307 6138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:38.460372 6138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:49:38.462834 6138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:39327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=41423
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:38.463140 6138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:38.463455 6138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:38.470687 6150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.470860 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.470687 6148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:38.472287 6138 server_base.cc:1034] running on GCE node
I20260812 14:49:38.472528 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:38.472855 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:38.474016 6138 hybrid_clock.cc:648] HybridClock initialized: now 1786546178474005 us; error 30 us; skew 500 ppm
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:38.477007 6138 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:49:38.477658 6138 webserver.cc:460] Webserver started at http://127.13.146.130:41423/ using document root <none> and password file <none>
I20260812 14:49:38.477994 6138 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:38.478152 6138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:38.498616 6138 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:38.499531 6170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.499773 6138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:38.499922 6138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "157e257ac5516f24e6976825321979e0"
server_key_iv: "5b6991b91000205851665aec895f879e"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:38.500475 6138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:38.554715 6138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:38.557487 6138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:38.557689 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:38.558002 6138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:38.558442 6138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:38.558497 6138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.558530 6138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:38.558550 6138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.567291 6138 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:39327
I20260812 14:49:38.567389 6400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:39327 every 8 connection(s)
I20260812 14:49:38.567831 6138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:49:38.571484 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 6138
I20260812 14:49:38.571874 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.146.148:47713/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:35173
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:44987
--webserver_port=41399
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:38.579859 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.568748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:43963 (local address 127.13.146.190:44987)
0812 14:49:38.568832 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:38.568834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:38.569135 (+ 301us) server_negotiation.cc:407] Connection header received
0812 14:49:38.569939 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:38.569940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:38.569955 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:38.570017 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:38.570846 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.571594 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.572767 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.572967 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.573045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.575924 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:38.575926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:38.575926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:38.575936 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:38.577922 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.578519 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.578520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.578521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.578591 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.578931 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.578933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.578933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.579119 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:38.579204 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:38.579635 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 14:49:38.579747 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:49:38.580754 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.568721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.130:43963)
0812 14:49:38.568917 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:38.568994 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:38.569798 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:38.570116 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:38.570121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:38.570396 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:38.570730 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.570738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.571707 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.571708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.572672 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.572674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.573100 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.573101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.573322 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.573908 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:38.573919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:38.575795 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:38.578032 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.578040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.578042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.578401 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.578674 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.578675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.578675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.578832 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.579242 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:38.579250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:38.579440 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 14:49:38.580289 (+ 849us) client_negotiation.cc:240] Negotiation successful
0812 14:49:38.580484 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260812 14:49:38.581584 6401 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:38.581758 6401 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.582037 6401 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.582775 6401 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:44987 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:49:38.582922 6401 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.583047 6401 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:38.583425 6401 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.583559 6401 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.583933 6401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:49:38.681594 6404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:38.681803 6404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:38.681828 6404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:38.682989 6404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:38.683033 6404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:38.683058 6404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:49:38.684613 6404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.146.148:47713/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:35173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=41399
--tserver_master_addrs=127.13.146.190:44987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:38.684854 6404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:38.685092 6404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:38.690788 6414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.690812 6415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:38.691285 6419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:38.691429 6404 server_base.cc:1034] running on GCE node
I20260812 14:49:38.691645 6404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:38.691946 6404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:38.693120 6404 hybrid_clock.cc:648] HybridClock initialized: now 1786546178693107 us; error 27 us; skew 500 ppm
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:38.696077 6404 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:49:38.696621 6404 webserver.cc:460] Webserver started at http://127.13.146.131:41399/ using document root <none> and password file <none>
I20260812 14:49:38.696878 6404 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:38.696962 6404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:38.719340 6404 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:38.721712 6433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.721952 6404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 14:49:38.722071 6404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "0baed1e7a9f345a5a34ffaee793b39fb"
format_stamp: "Formatted at 2026-08-12 14:49:37 on dist-test-slave-8hhm"
server_key: "342785a909dad4c48fdfbd911bf86431"
server_key_iv: "3d0a06997ec8ebedd117f0fdfefcb532"
server_key_version: "kuduclusterkey@0"
I20260812 14:49:38.722497 6404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:38.746747 6404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:38.749122 6404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:38.749346 6404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:38.749639 6404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:38.749996 6404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:38.750057 6404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.750095 6404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:38.750119 6404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:38.758543 6404 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:35173
I20260812 14:49:38.758718 6650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:35173 every 8 connection(s)
I20260812 14:49:38.759055 6404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:49:38.761292 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 6404
Aug 12 14:49:38 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546178, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:38.771658 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.759862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:44987 (local address 127.13.146.131:37079)
0812 14:49:38.760427 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:38.760473 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:38.761154 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:38.761659 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:38.761664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:38.761931 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:38.762212 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.762220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.763289 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.763290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.764615 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:38.764618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.765135 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.765137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:38.765305 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.765917 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:38.765928 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:38.767638 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:38.769921 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.769928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.769931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.770225 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.770439 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:38.770440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:38.770440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:38.770581 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:38.770937 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:38.770943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:38.771115 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 14:49:38.771229 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 14:49:38.771398 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":101,"threads_started":1}
I20260812 14:49:38.771673 6662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.761733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.13.146.190:44987)
0812 14:49:38.762556 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:38.762558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:38.762566 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:49:38.762600 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:38.762601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:38.762612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:38.763499 (+ 887us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:38.763533 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.764304 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.765951 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.766248 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.766318 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.768037 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:38.768038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:38.768039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:38.768045 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:38.769984 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.770663 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.770664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.770665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.770745 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.771012 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.771013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.771013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.771132 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:38.771316 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:38.771388 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:49:38.771480 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":70,"threads_started":1}
I20260812 14:49:38.772099 6137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:38.760191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:37079 (local address 127.13.146.190:44987)
0812 14:49:38.760244 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:38.760246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:38.760564 (+ 318us) server_negotiation.cc:407] Connection header received
0812 14:49:38.761329 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:38.761330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:38.761344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:38.761379 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:38.762337 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.763169 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.764695 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:38.764977 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:38.765074 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:38.767739 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:38.767740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:38.767741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:38.767749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:38.769826 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.770305 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.770306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.770307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.770365 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:38.770663 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:38.770664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:38.770665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:38.770848 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:38.770933 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:38.771942 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0812 14:49:38.772026 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:49:38.772570 6652 heartbeater.cc:347] Connected to a master server at 127.13.146.190:44987
I20260812 14:49:38.772696 6652 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.772912 6652 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.773825 6652 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:44987 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:49:38.773921 6652 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.774008 6652 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:38.774396 6652 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:38.774492 6652 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
W20260812 14:49:38.774919 6652 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 14:49:39 dist-test-slave-8hhm krb5kdc[3869](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786546179, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:39.342319 6133 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:39.342442 6133 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:39.342900 5735 ts_manager.cc:194] Registered new tserver with Master: b16ae72595ca4b0c9c04600a4b6198d6 (127.13.146.129:33099)
I20260812 14:49:39.343961 5735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:33003
I20260812 14:49:39.584177 6401 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:39.584357 6401 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:39.584645 5735 ts_manager.cc:194] Registered new tserver with Master: 3bab7c2a6c6544a1aa22b2649a1c03aa (127.13.146.130:39327)
I20260812 14:49:39.585188 5735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:43963
Aug 12 14:49:39 dist-test-slave-8hhm krb5kdc[3869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546179, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for HTTP/127.13.146.148@KRBTEST.COM
I20260812 14:49:39.775137 6652 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:39.775309 6652 heartbeater.cc:511] Master 127.13.146.190:44987 requested a full tablet report, sending...
I20260812 14:49:39.775655 5735 ts_manager.cc:194] Registered new tserver with Master: 0baed1e7a9f345a5a34ffaee793b39fb (127.13.146.131:35173)
I20260812 14:49:39.776196 5735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:37079
I20260812 14:49:39.779388 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:49:39.794015 6691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.780459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.13.146.190:44987)
0812 14:49:39.780613 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.780615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.780630 (+ 15us) server_negotiation.cc:407] Connection header received
0812 14:49:39.780717 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.780717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.780737 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.780823 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:39.781698 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.782448 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.783479 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.783824 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.783901 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.790336 (+ 6435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:39.790338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:39.790339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:39.790349 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:39.792293 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:39.792777 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:39.792778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:39.792779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:39.792855 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:39.793100 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:39.793100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:39.793101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:39.793283 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:39.793386 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.793639 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.793746 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260812 14:49:39.796280 5735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:49:39.798161 5735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:49:39.810091 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.803657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:39327 (local address 127.0.0.1:55812)
0812 14:49:39.805368 (+ 1711us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.805416 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:39.805511 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:39.805845 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:39.805848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:39.805851 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:39.805930 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.805932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.807382 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.807383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.808640 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.808643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.809765 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.809767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.809851 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.809865 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:49:39.809942 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:49:39.809998 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1675,"thread_start_us":138,"threads_started":1}
I20260812 14:49:39.810446 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.804000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.13.146.130:39327)
0812 14:49:39.804396 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.804403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.805508 (+ 1105us) server_negotiation.cc:407] Connection header received
0812 14:49:39.805615 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.805617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.805637 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.805729 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:39.806026 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.807253 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.808753 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.809560 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.809661 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.809723 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.810317 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.810366 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
I20260812 14:49:39.811299 6266 tablet_service.cc:1467] Processing CreateTablet for tablet fc62f99a1a124bce915845641d621ffc (DEFAULT_TABLE table=test-table [id=33835084618241f68c7b4c0303aff356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:39.811728 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62f99a1a124bce915845641d621ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:39.814191 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.803659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:35173 (local address 127.0.0.1:56026)
0812 14:49:39.805739 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.805749 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:39.805796 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:39.806844 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:39.806845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:39.806846 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:39.806879 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.806880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.808161 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.808162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.812370 (+ 4208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.812373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.813959 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.813960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.814021 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.814025 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:49:39.814073 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:49:39.814114 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2064,"thread_start_us":126,"threads_started":1}
I20260812 14:49:39.816077 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.803659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:33099 (local address 127.0.0.1:40740)
0812 14:49:39.805862 (+ 2203us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.805870 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:39.805904 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:39.806709 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:39.806710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:39.806712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:39.806770 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.806771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.809991 (+ 3220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.809992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.812931 (+ 2939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.812934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.815737 (+ 2803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.815738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.815808 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.815812 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:49:39.815858 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:49:39.815895 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2190,"mutex_wait_us":3,"thread_start_us":126,"threads_started":1}
I20260812 14:49:39.816490 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.803988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.13.146.131:35173)
0812 14:49:39.804223 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.804229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.805959 (+ 1730us) server_negotiation.cc:407] Connection header received
0812 14:49:39.806001 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.806003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.806019 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.806092 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:39.806962 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.808063 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.813026 (+ 4963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.813802 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.813873 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.813924 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.816357 (+ 2433us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.816410 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":67,"threads_started":1}
I20260812 14:49:39.816918 6711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.803771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.13.146.129:33099)
0812 14:49:39.805683 (+ 1912us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.805688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.806461 (+ 773us) server_negotiation.cc:407] Connection header received
0812 14:49:39.806503 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.806505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.806521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.806607 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:39.808788 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.809892 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.814441 (+ 4549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.815562 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.815647 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.815700 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.816791 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.816841 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":56,"threads_started":1}
I20260812 14:49:39.817400 6522 tablet_service.cc:1467] Processing CreateTablet for tablet fc62f99a1a124bce915845641d621ffc (DEFAULT_TABLE table=test-table [id=33835084618241f68c7b4c0303aff356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:39.817589 6003 tablet_service.cc:1467] Processing CreateTablet for tablet fc62f99a1a124bce915845641d621ffc (DEFAULT_TABLE table=test-table [id=33835084618241f68c7b4c0303aff356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:39.817835 6522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62f99a1a124bce915845641d621ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:39.817926 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62f99a1a124bce915845641d621ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:39.822230 6722 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: Bootstrap starting.
I20260812 14:49:39.822516 6717 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Bootstrap starting.
I20260812 14:49:39.823257 6717 tablet_bootstrap.cc:654] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:39.823274 6722 tablet_bootstrap.cc:654] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:39.823647 6717 log.cc:824] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:39.823920 6722 log.cc:824] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:39.824889 6722 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: No bootstrap required, opened a new log
I20260812 14:49:39.825037 6722 ts_tablet_manager.cc:1397] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:49:39.825433 6716 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Bootstrap starting.
I20260812 14:49:39.826221 6716 tablet_bootstrap.cc:654] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:39.826607 6716 log.cc:824] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:39.827646 6717 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: No bootstrap required, opened a new log
I20260812 14:49:39.827723 6717 ts_tablet_manager.cc:1397] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 14:49:39.828974 6722 raft_consensus.cc:348] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.829306 6722 raft_consensus.cc:374] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:39.829422 6722 raft_consensus.cc:729] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16ae72595ca4b0c9c04600a4b6198d6, State: Initialized, Role: FOLLOWER
I20260812 14:49:39.829645 6722 consensus_queue.cc:260] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.830062 6722 ts_tablet_manager.cc:1428] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 14:49:39.830053 6717 raft_consensus.cc:348] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.830217 6717 raft_consensus.cc:374] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:39.830255 6717 raft_consensus.cc:729] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0baed1e7a9f345a5a34ffaee793b39fb, State: Initialized, Role: FOLLOWER
I20260812 14:49:39.830360 6133 heartbeater.cc:503] Master 127.13.146.190:44987 was elected leader, sending a full tablet report...
I20260812 14:49:39.830358 6717 consensus_queue.cc:260] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.830626 6717 ts_tablet_manager.cc:1428] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:49:39.833350 6652 heartbeater.cc:503] Master 127.13.146.190:44987 was elected leader, sending a full tablet report...
I20260812 14:49:39.834664 6716 tablet_bootstrap.cc:492] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: No bootstrap required, opened a new log
I20260812 14:49:39.834748 6716 ts_tablet_manager.cc:1397] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260812 14:49:39.837559 6716 raft_consensus.cc:348] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.837752 6716 raft_consensus.cc:374] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:39.837790 6716 raft_consensus.cc:729] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bab7c2a6c6544a1aa22b2649a1c03aa, State: Initialized, Role: FOLLOWER
I20260812 14:49:39.837940 6716 consensus_queue.cc:260] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.838251 6716 ts_tablet_manager.cc:1428] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:49:39.838330 6401 heartbeater.cc:503] Master 127.13.146.190:44987 was elected leader, sending a full tablet report...
I20260812 14:49:39.875181 6730 raft_consensus.cc:482] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:39.875346 6730 raft_consensus.cc:504] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.875798 6730 leader_election.cc:288] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bab7c2a6c6544a1aa22b2649a1c03aa (127.13.146.130:39327), b16ae72595ca4b0c9c04600a4b6198d6 (127.13.146.129:33099)
I20260812 14:49:39.881008 6711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.876488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:45943 (local address 127.13.146.129:33099)
0812 14:49:39.876556 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.876558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.876857 (+ 299us) server_negotiation.cc:407] Connection header received
0812 14:49:39.876969 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.876971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.876981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.877013 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:39.877287 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.878006 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.879919 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.880551 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.880606 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.880615 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.880891 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.880934 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:49:39.881572 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.876145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:33099 (local address 127.13.146.131:45943)
0812 14:49:39.876787 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.876798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:39.876887 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:39.877087 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:39.877087 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:39.877089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:39.877166 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.877167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.878119 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.878120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.879768 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.879771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.880700 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.880701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.880775 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.880779 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:49:39.881371 (+ 592us) client_negotiation.cc:240] Negotiation successful
0812 14:49:39.881440 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":18,"thread_start_us":115,"threads_started":1}
I20260812 14:49:39.882135 6015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62f99a1a124bce915845641d621ffc" candidate_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" is_pre_election: true
I20260812 14:49:39.882316 6015 raft_consensus.cc:2454] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0baed1e7a9f345a5a34ffaee793b39fb in term 0.
I20260812 14:49:39.882679 6434 leader_election.cc:302] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0baed1e7a9f345a5a34ffaee793b39fb, b16ae72595ca4b0c9c04600a4b6198d6; no voters:
I20260812 14:49:39.882814 6730 raft_consensus.cc:2781] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:49:39.882860 6730 raft_consensus.cc:482] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:49:39.882886 6730 raft_consensus.cc:3037] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:39.884171 6730 raft_consensus.cc:504] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.884351 6730 leader_election.cc:288] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [CANDIDATE]: Term 1 election: Requested vote from peers 3bab7c2a6c6544a1aa22b2649a1c03aa (127.13.146.130:39327), b16ae72595ca4b0c9c04600a4b6198d6 (127.13.146.129:33099)
I20260812 14:49:39.884595 6015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62f99a1a124bce915845641d621ffc" candidate_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16ae72595ca4b0c9c04600a4b6198d6"
I20260812 14:49:39.884698 6015 raft_consensus.cc:3037] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:39.885703 6015 raft_consensus.cc:2454] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0baed1e7a9f345a5a34ffaee793b39fb in term 1.
I20260812 14:49:39.885898 6434 leader_election.cc:302] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0baed1e7a9f345a5a34ffaee793b39fb, b16ae72595ca4b0c9c04600a4b6198d6; no voters:
I20260812 14:49:39.885990 6730 raft_consensus.cc:2781] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:39.886138 6730 raft_consensus.cc:686] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 1 LEADER]: Becoming Leader. State: Replica: 0baed1e7a9f345a5a34ffaee793b39fb, State: Running, Role: LEADER
I20260812 14:49:39.886260 6730 consensus_queue.cc:237] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } }
I20260812 14:49:39.887198 5733 catalog_manager.cc:5495] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0baed1e7a9f345a5a34ffaee793b39fb (127.13.146.131). New cstate: current_term: 1 leader_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35173 } health_report { overall_health: HEALTHY } } }
I20260812 14:49:39.888919 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.876515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:46513 (local address 127.13.146.130:39327)
0812 14:49:39.876568 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.876571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.876583 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:49:39.876618 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.876619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.876630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.876699 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:39.876958 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.877712 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.882310 (+ 4598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.882959 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.883015 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.883026 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.888745 (+ 5719us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.888814 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:49:39.889267 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.876054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:39327 (local address 127.13.146.131:46513)
0812 14:49:39.876278 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.876290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:39.876348 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:39.876764 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:39.876766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:39.876769 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:39.876862 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.876865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.877841 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.877843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.881433 (+ 3590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:39.881437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.883093 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.883094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:39.888622 (+ 5528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.888628 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:49:39.889058 (+ 430us) client_negotiation.cc:240] Negotiation successful
0812 14:49:39.889125 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":113,"threads_started":1}
I20260812 14:49:39.889678 6303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62f99a1a124bce915845641d621ffc" candidate_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa"
I20260812 14:49:39.889674 6306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62f99a1a124bce915845641d621ffc" candidate_uuid: "0baed1e7a9f345a5a34ffaee793b39fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" is_pre_election: true
I20260812 14:49:39.889890 6303 raft_consensus.cc:3037] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:39.891060 6303 raft_consensus.cc:2454] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0baed1e7a9f345a5a34ffaee793b39fb in term 1.
I20260812 14:49:39.904708 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.900376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.13.146.131:35173)
0812 14:49:39.900420 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:39.900422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:39.900429 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:49:39.900465 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:39.900466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:39.900487 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:39.900550 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:49:39.900751 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.901660 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.903289 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:39.903710 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:39.903796 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:39.904038 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:49:39.904199 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:49:39.904295 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:39.904551 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 14:49:39.904591 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:49:39.907251 6015 raft_consensus.cc:1261] T fc62f99a1a124bce915845641d621ffc P b16ae72595ca4b0c9c04600a4b6198d6 [term 1 FOLLOWER]: Refusing update from remote peer 0baed1e7a9f345a5a34ffaee793b39fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:39.907289 6303 raft_consensus.cc:1261] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 1 FOLLOWER]: Refusing update from remote peer 0baed1e7a9f345a5a34ffaee793b39fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:39.907585 6743 consensus_queue.cc:1035] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16ae72595ca4b0c9c04600a4b6198d6" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:39.907780 6730 consensus_queue.cc:1035] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bab7c2a6c6544a1aa22b2649a1c03aa" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:39.910413 6746 mvcc.cc:205] Tried to move back new op lower bound from 7317693152897556480 to 7317693152814788608. Current Snapshot: MvccSnapshot[applied={T|T < 7317693152897556480}]
I20260812 14:49:39.912940 6748 mvcc.cc:205] Tried to move back new op lower bound from 7317693152897556480 to 7317693152814788608. Current Snapshot: MvccSnapshot[applied={T|T < 7317693152897556480}]
I20260812 14:49:39.916503 5733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:39.916615 5733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:39.917529 5733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3c73991d9e349fca084135fa1e8145e: Sending DeleteTablet for 3 replicas of tablet fc62f99a1a124bce915845641d621ffc
I20260812 14:49:39.917995 6266 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62f99a1a124bce915845641d621ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:39 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:55812
I20260812 14:49:39.918236 6522 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62f99a1a124bce915845641d621ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:39 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:56026
I20260812 14:49:39.918609 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62f99a1a124bce915845641d621ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:39 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:40740
I20260812 14:49:39.919261 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 5817
I20260812 14:49:39.921321 6764 tablet_replica.cc:330] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: stopping tablet replica
I20260812 14:49:39.921443 6764 raft_consensus.cc:2229] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:49:39.921569 6764 raft_consensus.cc:2258] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:39.924428 6766 tablet_replica.cc:330] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: stopping tablet replica
I20260812 14:49:39.924549 6766 raft_consensus.cc:2229] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:49:39.924652 6766 raft_consensus.cc:2258] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:39.933771 6764 ts_tablet_manager.cc:1905] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:39.936931 6764 ts_tablet_manager.cc:1918] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:39.937037 6764 log.cc:1196] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/fc62f99a1a124bce915845641d621ffc
I20260812 14:49:39.937378 6764 ts_tablet_manager.cc:1939] T fc62f99a1a124bce915845641d621ffc P 3bab7c2a6c6544a1aa22b2649a1c03aa: Deleting consensus metadata
I20260812 14:49:39.937930 5705 catalog_manager.cc:4841] TS 3bab7c2a6c6544a1aa22b2649a1c03aa (127.13.146.130:39327): tablet fc62f99a1a124bce915845641d621ffc (table test-table [id=33835084618241f68c7b4c0303aff356]) successfully deleted
I20260812 14:49:39.938525 6766 ts_tablet_manager.cc:1905] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:39.946118 6766 ts_tablet_manager.cc:1918] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:39.946275 6766 log.cc:1196] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/fc62f99a1a124bce915845641d621ffc
I20260812 14:49:39.946606 6766 ts_tablet_manager.cc:1939] T fc62f99a1a124bce915845641d621ffc P 0baed1e7a9f345a5a34ffaee793b39fb: Deleting consensus metadata
I20260812 14:49:39.947278 5708 catalog_manager.cc:4841] TS 0baed1e7a9f345a5a34ffaee793b39fb (127.13.146.131:35173): tablet fc62f99a1a124bce915845641d621ffc (table test-table [id=33835084618241f68c7b4c0303aff356]) successfully deleted
W20260812 14:49:39.947516 5703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:33099 (error 108)
I20260812 14:49:39.948184 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 6138
I20260812 14:49:39.977939 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:39.977738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:33099 (local address 127.0.0.1:40752)
0812 14:49:39.977800 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:39.977875 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:49:39.978221 5703 catalog_manager.cc:4584] TS b16ae72595ca4b0c9c04600a4b6198d6 (127.13.146.129:33099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc62f99a1a124bce915845641d621ffc: Network error: Client connection negotiation failed: client connection to 127.13.146.129:33099: connect: Connection refused (error 111)
I20260812 14:49:40.031050 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:40.030878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:33099 (local address 127.0.0.1:40758)
0812 14:49:40.030950 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:40.030994 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 14:49:40.031657 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:40.031548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:33099 (local address 127.0.0.1:40764)
0812 14:49:40.031594 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:40.031621 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:33099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:49:40.053660 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 6404
I20260812 14:49:40.067525 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 5669
I20260812 14:49:40.083426 13898 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:49:40.428908 13898 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:49:40.429009 13898 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:49:40.784176 13898 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:49:40.784 UTC [3935] LOG: received smart shutdown request
2026-08-12 14:49:40.788 UTC [3935] LOG: background worker "logical replication launcher" (PID 3948) exited with exit code 1
2026-08-12 14:49:40.788 UTC [3943] LOG: shutting down
2026-08-12 14:49:40.966 UTC [3935] LOG: database system is shut down
2026-08-12T14:49:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50006 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:49:41 dist-test-slave-8hhm krb5kdc[6783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:49:41 dist-test-slave-8hhm krb5kdc[6783](info): set up 2 sockets
Aug 12 14:49:41 dist-test-slave-8hhm krb5kdc[6783](info): commencing operation
krb5kdc: starting...
W20260812 14:49:42.433701 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:49:42.564805 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34599
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:42.648463 6803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:42.648689 6803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:42.648720 6803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:42.649943 6803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:42.649989 6803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:42.650004 6803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:42.650017 6803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:42.650029 6803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:42.651602 6803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:42.651907 6803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:42.652159 6803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:42.657238 6812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:42.657239 6814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:42.657260 6818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:42.657789 6803 server_base.cc:1034] running on GCE node
I20260812 14:49:42.658123 6803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:42.658419 6803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:42.659595 6803 hybrid_clock.cc:648] HybridClock initialized: now 1786546182659595 us; error 50 us; skew 500 ppm
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:42.667418 6803 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:42.667963 6803 webserver.cc:460] Webserver started at http://127.13.146.190:42509/ using document root <none> and password file <none>
I20260812 14:49:42.668223 6803 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:42.668298 6803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:42.668466 6803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:42.669663 6803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "89413283058f4cc8a41fb2775379c92c"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "5337a61a59ff1ddc7d5a850c4d606dba"
server_key_iv: "84eee03e9d3982323783c83be090cc42"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.670070 6803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "89413283058f4cc8a41fb2775379c92c"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "5337a61a59ff1ddc7d5a850c4d606dba"
server_key_iv: "84eee03e9d3982323783c83be090cc42"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.671463 6803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:49:42.672286 6829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:42.672498 6803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:42.672613 6803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "89413283058f4cc8a41fb2775379c92c"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "5337a61a59ff1ddc7d5a850c4d606dba"
server_key_iv: "84eee03e9d3982323783c83be090cc42"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.672693 6803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:42.695123 6803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:42.697712 6803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:42.697890 6803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:42.702247 6803 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34599
I20260812 14:49:42.702306 6937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34599 every 8 connection(s)
I20260812 14:49:42.702694 6803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:42.703696 6938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:42.705338 6938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:49:42.710796 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 6803
I20260812 14:49:42.710943 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:49:42.711210 13898 external_mini_cluster.cc:1428] Setting key 791d8c3073d537f65770af26674a4790
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:42.715714 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.706014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34599 (local address 127.0.0.1:44914)
0812 14:49:42.706206 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.706246 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:42.706740 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:42.707038 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:42.707041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:42.707322 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:42.707482 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.707487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.708506 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.708507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.709250 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.709252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.709751 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.709752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.709817 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.710331 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:42.710338 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:42.712020 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:42.714186 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.714193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.714195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.714410 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.714692 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.714692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.714693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.714780 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.715232 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:42.715237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:42.715297 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 14:49:42.715374 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:49:42.715506 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
I20260812 14:49:42.715785 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.706175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.13.146.190:34599)
0812 14:49:42.706308 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:42.706312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:42.706352 (+ 40us) server_negotiation.cc:407] Connection header received
0812 14:49:42.706884 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:42.706885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:42.706903 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:42.706982 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:42.707586 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.708398 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.709356 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.709666 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.709754 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.712113 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:42.712120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:42.712120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:42.712127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:42.714079 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.714505 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.714508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.714508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.714618 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.714851 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.714852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.714852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.715155 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:42.715251 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:42.715438 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 14:49:42.715600 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260812 14:49:42.716639 6938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:49:42.717049 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.716904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53598)
0812 14:49:42.716984 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.717029 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 14:49:42.717243 6834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
I20260812 14:49:42.717680 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.717589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53612)
0812 14:49:42.717637 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.717663 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:49:42.717830 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:44305: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
I20260812 14:49:42.739861 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:44305
--webserver_interface=127.13.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:42.767086 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } attempt: 1
I20260812 14:49:42.767635 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.767463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53624)
0812 14:49:42.767521 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.767588 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:49:42.768092 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.767994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53634)
0812 14:49:42.768034 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.768066 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:49:42.768188 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:44305: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
W20260812 14:49:42.827975 6952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:42.828233 6952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:42.828274 6952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:42.829486 6952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:42.829566 6952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:42.829591 6952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:42.829612 6952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:42.829643 6952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:42.831199 6952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:44305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:42.831506 6952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:42.831764 6952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:49:42.836333 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } attempt: 2
I20260812 14:49:42.836822 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.836615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53650)
0812 14:49:42.836742 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.836781 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260812 14:49:42.837003 6971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:42.837021 6966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:42.837092 6961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:42.837352 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.837237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53656)
0812 14:49:42.837298 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.837323 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 14:49:42.837458 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:44305: Network error: Client connection negotiation failed: client connection to 127.13.146.189:44305: connect: Connection refused (error 111)
I20260812 14:49:42.837607 6952 server_base.cc:1034] running on GCE node
I20260812 14:49:42.837895 6952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:42.838201 6952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:42.839394 6952 hybrid_clock.cc:648] HybridClock initialized: now 1786546182839373 us; error 37 us; skew 500 ppm
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:42.842038 6952 init.cc:377] Logged in from keytab as kudu/127.13.146.189@KRBTEST.COM (short username kudu)
I20260812 14:49:42.842545 6952 webserver.cc:460] Webserver started at http://127.13.146.189:46663/ using document root <none> and password file <none>
I20260812 14:49:42.842792 6952 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:42.842865 6952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:42.843021 6952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:42.844126 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data/instance:
uuid: "648b7f0c387e4ea98cd4628a1fea8e06"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "2b8713b17fb5254829bf4e8ecefd6e9e"
server_key_iv: "1963467b48f579fe58bf4efaeb2a81f1"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.844516 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal/instance:
uuid: "648b7f0c387e4ea98cd4628a1fea8e06"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "2b8713b17fb5254829bf4e8ecefd6e9e"
server_key_iv: "1963467b48f579fe58bf4efaeb2a81f1"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.846136 6952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:49:42.846897 6985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:42.847079 6952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:49:42.847193 6952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "648b7f0c387e4ea98cd4628a1fea8e06"
format_stamp: "Formatted at 2026-08-12 14:49:42 on dist-test-slave-8hhm"
server_key: "2b8713b17fb5254829bf4e8ecefd6e9e"
server_key_iv: "1963467b48f579fe58bf4efaeb2a81f1"
server_key_version: "encryptionkey@0"
I20260812 14:49:42.847273 6952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:42.858436 6952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:42.860291 6952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:42.860459 6952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:42.864568 6952 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.189:44305
I20260812 14:49:42.864609 7083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.189:44305 every 8 connection(s)
I20260812 14:49:42.864995 6952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
I20260812 14:49:42.865659 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 6952
I20260812 14:49:42.865792 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-1/wal/instance
I20260812 14:49:42.865978 13898 external_mini_cluster.cc:1428] Setting key 01ad399b559f0f62039564a4e4d744b4
I20260812 14:49:42.866030 7084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:42.867805 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.188@KRBTEST.COM" created.
I20260812 14:49:42.877910 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.868713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.13.146.190:34599)
0812 14:49:42.868790 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:42.868793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:42.869003 (+ 210us) server_negotiation.cc:407] Connection header received
0812 14:49:42.869609 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:42.869611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:42.869628 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:42.869687 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:42.870350 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.870850 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.871865 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.872084 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.872162 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.874657 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:42.874659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:42.874659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:42.874674 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:42.876326 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.876778 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.876779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.876780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.876843 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.877119 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.877120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.877120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.877317 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:42.877388 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:42.877733 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 14:49:42.877829 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:49:42.878166 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.868639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34599 (local address 127.0.0.1:44918)
0812 14:49:42.868862 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.868904 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:42.869492 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:42.869760 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:42.869764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:42.870030 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:42.870235 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.870239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.870951 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.870952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.871765 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.871768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.872184 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.872185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.872319 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.872887 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:42.872895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:42.874546 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:42.876414 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.876423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.876425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.876688 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.876929 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.876930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.876930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.877042 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.877392 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:42.877399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:42.877570 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 14:49:42.877679 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:49:42.877853 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260812 14:49:42.878708 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:49:42.888051 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.878984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53660)
0812 14:49:42.879044 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.879063 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:42.879110 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:42.879351 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:42.879352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:42.879493 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:42.879537 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.879538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.880503 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.880504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.881384 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.881388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.882187 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.882188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.882259 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.882708 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:42.882714 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:42.884313 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:42.886573 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.886574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.886574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.886850 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.887128 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.887129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.887129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.887207 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.887664 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:42.887666 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:42.887728 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 14:49:42.887808 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:49:42.887913 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:49:42.888265 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.879022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.13.146.189:44305)
0812 14:49:42.879190 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:42.879195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:42.879213 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:49:42.879241 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:42.879242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:42.879253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:42.879335 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:42.879630 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.880431 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.881493 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.881989 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.882102 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.884413 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:42.884422 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:42.884422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:42.884428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:42.886471 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.886955 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.886959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.886959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.887021 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.887288 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.887289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.887289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.887479 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:42.887564 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:42.888007 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 14:49:42.888123 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260812 14:49:42.893397 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:49:42.893782 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.893665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50126)
0812 14:49:42.893708 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.893755 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 14:49:42.893908 6986 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
I20260812 14:49:42.895918 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.895834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50140)
0812 14:49:42.895865 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.895891 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 14:49:42.896086 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
I20260812 14:49:42.903872 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:43389
--webserver_interface=127.13.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:42.928597 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } attempt: 3
Aug 12 14:49:42 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:49:42.936127 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.929057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53672 (local address 127.13.146.189:44305)
0812 14:49:42.929128 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:42.929130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:42.929141 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:49:42.929174 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:42.929175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:42.929185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:42.929289 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:42.929696 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.930191 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.931188 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.931390 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.931440 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.933472 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:42.933473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:42.933474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:42.933481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:42.934939 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.935320 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.935321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.935322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.935380 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:42.935586 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:42.935587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:42.935587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:42.935703 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:42.935778 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:42.935953 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:49:42.936031 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:49:42.936127 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.928976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53672)
0812 14:49:42.929024 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.929037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:42.929113 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:42.929314 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:42.929315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:42.929508 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:42.929595 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.929597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.930292 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.930293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.931088 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:42.931091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:42.931459 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:42.931460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:42.931536 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:42.931998 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:42.932004 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:42.933382 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:42.935045 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.935046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.935047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.935231 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.935450 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:42.935451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:42.935451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:42.935513 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:42.935781 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:42.935783 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:42.935831 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:49:42.935911 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:49:42.936021 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:49:42.936507 6938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:49:42.936889 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.936786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50154)
0812 14:49:42.936838 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.936869 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:49:42.937177 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.937123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50170)
0812 14:49:42.937146 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.937166 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 14:49:42.937300 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
I20260812 14:49:42.945400 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 1
I20260812 14:49:42.945814 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.945682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50172)
0812 14:49:42.945740 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.945780 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:49:42.946183 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.946084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50180)
0812 14:49:42.946132 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.946161 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:49:42.946285 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
I20260812 14:49:42.968490 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 1
I20260812 14:49:42.969095 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.968936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50186)
0812 14:49:42.969007 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.969058 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 14:49:42.969519 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:42.969412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50200)
0812 14:49:42.969465 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:42.969495 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 14:49:42.969657 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
W20260812 14:49:42.997095 7092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:42.997373 7092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:42.997416 7092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:42.998562 7092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:42.998631 7092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:42.998656 7092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:42.998684 7092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:42.998715 7092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:43.000207 7092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34599,127.13.146.189:44305,127.13.146.188:43389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:43389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:43.000494 7092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:43.000728 7092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:43.005761 7107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:43.005789 7103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:43.005789 7102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:43.006071 7092 server_base.cc:1034] running on GCE node
I20260812 14:49:43.006340 7092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:43.006649 7092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:43.007870 7092 hybrid_clock.cc:648] HybridClock initialized: now 1786546183007853 us; error 51 us; skew 500 ppm
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546183, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:43.010571 7092 init.cc:377] Logged in from keytab as kudu/127.13.146.188@KRBTEST.COM (short username kudu)
I20260812 14:49:43.011040 7092 webserver.cc:460] Webserver started at http://127.13.146.188:39123/ using document root <none> and password file <none>
I20260812 14:49:43.011294 7092 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:43.011368 7092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:43.011528 7092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:43.012676 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data/instance:
uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c"
format_stamp: "Formatted at 2026-08-12 14:49:43 on dist-test-slave-8hhm"
server_key: "7e7db510ecb118c8e097e5099ea5a1dc"
server_key_iv: "4f844c4b8515b05ef382e07c57c0000c"
server_key_version: "encryptionkey@0"
I20260812 14:49:43.013078 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal/instance:
uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c"
format_stamp: "Formatted at 2026-08-12 14:49:43 on dist-test-slave-8hhm"
server_key: "7e7db510ecb118c8e097e5099ea5a1dc"
server_key_iv: "4f844c4b8515b05ef382e07c57c0000c"
server_key_version: "encryptionkey@0"
I20260812 14:49:43.014439 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 2
I20260812 14:49:43.014493 7092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:43.014835 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.014688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50210)
0812 14:49:43.014758 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.014800 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 14:49:43.015255 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.015119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50218)
0812 14:49:43.015175 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.015222 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 14:49:43.015353 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
I20260812 14:49:43.015487 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:43.015652 7092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:43.015754 7092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c"
format_stamp: "Formatted at 2026-08-12 14:49:43 on dist-test-slave-8hhm"
server_key: "7e7db510ecb118c8e097e5099ea5a1dc"
server_key_iv: "4f844c4b8515b05ef382e07c57c0000c"
server_key_version: "encryptionkey@0"
I20260812 14:49:43.015811 7092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:43.041319 7092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:43.043368 7092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:43.043488 7092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:43.044823 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 2
I20260812 14:49:43.045313 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.045105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50220)
0812 14:49:43.045166 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.045255 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:49:43.045724 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.045584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50234)
0812 14:49:43.045635 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.045666 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:49:43.045969 6938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:43389: Network error: Client connection negotiation failed: client connection to 127.13.146.188:43389: connect: Connection refused (error 111)
I20260812 14:49:43.048169 7092 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.188:43389
I20260812 14:49:43.048230 7216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.188:43389 every 8 connection(s)
I20260812 14:49:43.048585 7092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
I20260812 14:49:43.049602 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:43.051234 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 7092
I20260812 14:49:43.051374 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786546020513802-13898-0/minicluster-data/master-2/wal/instance
I20260812 14:49:43.051566 13898 external_mini_cluster.cc:1428] Setting key 54579f3ac69b32e2cabdcf23b48f8bf6
I20260812 14:49:43.051630 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546183, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:49:43.061328 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.052440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.13.146.190:34599)
0812 14:49:43.052489 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.052491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.052646 (+ 155us) server_negotiation.cc:407] Connection header received
0812 14:49:43.052921 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.052922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.052935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.052995 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.053638 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.054164 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.055078 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.055288 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.055362 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.057784 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.057786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.057786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.057794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.059964 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060375 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.060376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.060376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.060445 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060683 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.060684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.060684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.060813 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.060894 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.061100 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.061193 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:49:43.061523 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.052364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34599 (local address 127.0.0.1:44922)
0812 14:49:43.052523 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.052573 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:43.052828 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:43.053017 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:43.053020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:43.053374 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:43.053539 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.053544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.054255 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.054256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.054970 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.054972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.055386 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.055387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.055512 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.056021 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:43.056029 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:43.057677 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:43.060057 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060065 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.060068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.060292 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.060516 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.060518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.060613 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.060885 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:43.060894 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:43.060968 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 14:49:43.061064 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 14:49:43.061271 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260812 14:49:43.061801 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.052085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.13.146.188:43389)
0812 14:49:43.052262 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.052272 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.052305 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:49:43.052815 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.052822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.052845 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.052958 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.053649 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.054432 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.055197 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.055514 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.055596 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.058236 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.058244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.058245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.058252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.060004 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060397 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.060400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.060400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.060487 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.060702 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.060703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.060703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.060928 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.061078 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.061161 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.061351 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260812 14:49:43.062132 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546183, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:43.069658 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.062502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53682 (local address 127.13.146.189:44305)
0812 14:49:43.062550 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.062552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.062560 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:49:43.062584 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.062585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.062595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.062684 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.063040 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.063517 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.064200 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.064414 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.064491 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.066628 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.066630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.066630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.066637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.068430 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.068787 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.068788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.068789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.068852 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.069065 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.069066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.069067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.069185 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.069322 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.069491 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.069567 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:49:43.069677 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.062417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:44305 (local address 127.0.0.1:53682)
0812 14:49:43.062468 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.062480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:43.062519 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:43.062701 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:43.062702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:43.062906 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:43.062949 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.062951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.063602 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.063603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.064105 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.064107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.064486 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.064487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.064555 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.065024 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:43.065030 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:43.066534 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:43.068522 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.068523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.068524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.068701 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.068923 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.068924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.068924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.068992 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.069316 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:43.069317 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:43.069365 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:49:43.069448 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 14:49:43.069556 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:49:43.070096 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:49:43.070495 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.062809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.13.146.190:34599)
0812 14:49:43.062870 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.062873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.062880 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:49:43.062900 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.062901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.062909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.062954 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.063442 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.063901 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.064577 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.064871 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.064941 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.067270 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.067272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.067273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.067279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.068936 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.069399 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.069400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.069400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.069462 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.069693 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.069695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.069695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.069890 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.070076 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.070293 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.070385 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546183, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:49:43.077591 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.070381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50254)
0812 14:49:43.070625 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.070637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:43.070693 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:43.070824 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:43.070824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:43.070982 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:43.071019 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.071021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.071750 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.071751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.072217 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.072218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.072584 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.072585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.072631 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.073076 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:43.073081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:43.074318 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:43.076237 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.076238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.076238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.076505 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.076711 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.076712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.076712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.076798 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.077117 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:43.077118 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:43.077163 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:49:43.077292 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 14:49:43.077418 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 14:49:43.077591 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.070448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.13.146.188:43389)
0812 14:49:43.070494 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.070496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.070716 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:49:43.070752 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.070753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.070765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.070822 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.071105 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.071655 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.072316 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.072515 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.072587 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.074405 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.074407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.074407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.074414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.076143 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.076563 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.076564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.076565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.076634 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.076873 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.076873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.076874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.076994 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.077123 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.077327 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.077475 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":23}
I20260812 14:49:43.079175 7220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: Bootstrap starting.
I20260812 14:49:43.079331 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.071155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53686 (local address 127.13.146.189:44305)
0812 14:49:43.071219 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.071221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.071229 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:49:43.071248 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.071249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.071259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.071314 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.071803 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.072247 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.072971 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.073309 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.073381 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.076245 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.076247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.076247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.076254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.078090 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.078443 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.078445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.078445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.078507 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.078744 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.078746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.078746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.078872 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.079095 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.079137 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.079226 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:49:43.080384 7220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:43.080713 7220 log.cc:824] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:43.081187 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:49:43.081676 7220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: No bootstrap required, opened a new log
I20260812 14:49:43.083873 7220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.084056 7220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:43.084095 7220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71cd3a3ec6e64128ae9ffba06c9b6f6c, State: Initialized, Role: FOLLOWER
I20260812 14:49:43.084223 7220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.084506 7220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: configured and running, proceeding with master startup.
W20260812 14:49:43.086642 7256 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:49:43.086704 7256 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:49:43.086901 7092 master_runner.cc:186] Error getting master registration for 127.13.146.190:34599: OK, instance_id { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" instance_seqno: 1786546182700965 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:49:43.089442 7092 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:49:43.092649 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.082783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.13.146.188:43389)
0812 14:49:43.082824 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.082825 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.083086 (+ 261us) server_negotiation.cc:407] Connection header received
0812 14:49:43.083180 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.083181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.083190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.083221 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.084034 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.084717 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086149 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.086412 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086458 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.088192 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.088193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.088194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.088201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.090173 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.091629 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.091630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.091630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.091696 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.091961 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.091962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.091963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.092140 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.092222 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.092474 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.092560 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:49:43.092691 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.083028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.13.146.190:34599)
0812 14:49:43.083073 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.083075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.083317 (+ 242us) server_negotiation.cc:407] Connection header received
0812 14:49:43.083423 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.083424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.083437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.083472 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.084288 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.085050 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086087 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.086405 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086447 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.088768 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.088770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.088771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.088781 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.090627 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.091763 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.091764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.091765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.091850 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.092081 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.092082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.092082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.092220 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.092283 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.092515 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.092596 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:49:43.092691 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.082794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.13.146.189:44305)
0812 14:49:43.082840 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.082843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.082961 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:49:43.083091 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.083093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.083105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.083139 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.084056 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.084846 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086045 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.086357 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.086401 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.089141 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.089143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.089144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.089152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.091059 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.091770 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.091771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.091771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.091851 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.092087 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.092088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.092088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.092208 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.092281 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.092515 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.092596 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:49:43.097496 7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.097630 7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:49:43.106501 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 3
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:49:43.114040 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.106903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.13.146.188:43389)
0812 14:49:43.106969 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.106971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.106980 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:49:43.107039 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.107040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.107050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.107104 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.107409 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.107972 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.108750 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.108931 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.109008 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.111262 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.111263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.111264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.111270 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.112792 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.113194 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.113195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.113196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.113304 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.113515 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.113516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.113516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.113632 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.113717 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.113868 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.113942 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:49:43.114040 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.106824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50278)
0812 14:49:43.106900 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.106910 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:43.106975 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:43.107113 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:43.107114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:43.107281 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:43.107321 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.107323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.108064 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.108065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.108644 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.108646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.109005 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.109006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.109058 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.109570 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:43.109582 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:43.111145 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:43.112880 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.112881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.112881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.113117 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.113383 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.113384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.113384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.113447 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.113703 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:43.113704 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:43.113749 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:49:43.113841 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:49:43.113942 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 14:49:43.115406 7084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: Bootstrap starting.
I20260812 14:49:43.116135 7084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:43.116432 7084 log.cc:824] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:43.117270 7084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: No bootstrap required, opened a new log
I20260812 14:49:43.119238 7084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.119376 7084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:43.119424 7084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648b7f0c387e4ea98cd4628a1fea8e06, State: Initialized, Role: FOLLOWER
I20260812 14:49:43.119570 7084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.119899 7084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:43.119903 7259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.120044 7259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:49:43.121763 7282 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:49:43.121876 7282 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:49:43.122007 6952 master_runner.cc:186] Error getting master registration for 127.13.146.190:34599: OK, instance_id { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" instance_seqno: 1786546182700965 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:49:43.124466 6952 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:49:43.145150 6938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } attempt: 3
Aug 12 14:49:43 dist-test-slave-8hhm krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546182, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:49:43.154611 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.145504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:43389 (local address 127.0.0.1:50284)
0812 14:49:43.145555 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:43.145566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:43.145607 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:43.145775 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:43.145776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:43.145991 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:43.146043 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.146045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.147085 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.147086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.147929 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:43.147931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.148488 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.148490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:43.148551 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.149140 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:43.149146 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:43.150946 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:43.153425 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.153426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.153427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.153655 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.153890 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:43.153891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:43.153891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:43.153963 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:43.154273 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:43.154274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:43.154325 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 14:49:43.154400 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:49:43.154504 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:49:43.154642 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:43.145576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.13.146.188:43389)
0812 14:49:43.145631 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:43.145633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:43.145645 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:49:43.145670 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:43.145671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:43.145689 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:43.145769 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:43.146146 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.146982 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.148040 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:43.148392 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:43.148478 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:43.151326 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:43.151328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:43.151328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:43.151337 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:43.153331 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.153741 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.153742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.153743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.153812 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:43.154044 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:43.154045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:43.154046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:43.154197 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:43.154273 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:43.154446 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 14:49:43.154539 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:49:43.156864 6938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: Bootstrap starting.
I20260812 14:49:43.157783 6938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:43.158103 6938 log.cc:824] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:43.159078 6938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: No bootstrap required, opened a new log
I20260812 14:49:43.161128 6938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.161312 6938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:43.161351 6938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89413283058f4cc8a41fb2775379c92c, State: Initialized, Role: FOLLOWER
I20260812 14:49:43.161481 6938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.161803 6938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:43.161769 7290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.161983 7290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:49:43.163717 7312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:49:43.163872 7312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:49:43.166496 6803 master_runner.cc:263] No leader master found from master 89413283058f4cc8a41fb2775379c92c
I20260812 14:49:43.166553 6803 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 14:49:43.198771 6837 tablet.cc:2369] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:49:43.294122 7134 tablet.cc:2369] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:49:43.310761 7259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:43.310894 7259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.311195 7259 leader_election.cc:288] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89413283058f4cc8a41fb2775379c92c (127.13.146.190:34599), 71cd3a3ec6e64128ae9ffba06c9b6f6c (127.13.146.188:43389)
I20260812 14:49:43.311537 6885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89413283058f4cc8a41fb2775379c92c" is_pre_election: true
I20260812 14:49:43.311596 7176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" is_pre_election: true
I20260812 14:49:43.311710 6885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648b7f0c387e4ea98cd4628a1fea8e06 in term 0.
I20260812 14:49:43.311830 7176 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648b7f0c387e4ea98cd4628a1fea8e06 in term 0.
I20260812 14:49:43.311951 6991 leader_election.cc:302] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648b7f0c387e4ea98cd4628a1fea8e06, 89413283058f4cc8a41fb2775379c92c; no voters:
I20260812 14:49:43.312125 7259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:49:43.312182 7259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:49:43.312232 7259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:43.312963 7259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.313122 7259 leader_election.cc:288] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [CANDIDATE]: Term 1 election: Requested vote from peers 89413283058f4cc8a41fb2775379c92c (127.13.146.190:34599), 71cd3a3ec6e64128ae9ffba06c9b6f6c (127.13.146.188:43389)
I20260812 14:49:43.313380 6885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89413283058f4cc8a41fb2775379c92c"
I20260812 14:49:43.313380 7176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c"
I20260812 14:49:43.313468 6885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:43.313469 7176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:43.314349 6885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648b7f0c387e4ea98cd4628a1fea8e06 in term 1.
I20260812 14:49:43.314349 7176 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648b7f0c387e4ea98cd4628a1fea8e06 in term 1.
I20260812 14:49:43.314536 6986 leader_election.cc:302] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648b7f0c387e4ea98cd4628a1fea8e06, 71cd3a3ec6e64128ae9ffba06c9b6f6c; no voters:
I20260812 14:49:43.314621 7259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:43.314805 7259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [term 1 LEADER]: Becoming Leader. State: Replica: 648b7f0c387e4ea98cd4628a1fea8e06, State: Running, Role: LEADER
I20260812 14:49:43.314908 7259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } }
I20260812 14:49:43.315380 7314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648b7f0c387e4ea98cd4628a1fea8e06. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.315490 7314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:43.315732 7316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:43.315909 7316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:43.317353 6885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [term 1 FOLLOWER]: Refusing update from remote peer 648b7f0c387e4ea98cd4628a1fea8e06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:43.317342 7176 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [term 1 FOLLOWER]: Refusing update from remote peer 648b7f0c387e4ea98cd4628a1fea8e06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:43.317682 7314 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:43.317806 7259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:43.319079 7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648b7f0c387e4ea98cd4628a1fea8e06. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.319204 7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:49:43.319226 7290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648b7f0c387e4ea98cd4628a1fea8e06. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.319337 7290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:49:43.320273 7259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.320362 7259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:43.320406 7259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.320443 7259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:43.320506 7317 mvcc.cc:205] Tried to move back new op lower bound from 7317693166865256448 to 7317693166858350592. Current Snapshot: MvccSnapshot[applied={T|T < 7317693166865256448}]
I20260812 14:49:43.320613 7290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.320709 7290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:49:43.320652 7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89413283058f4cc8a41fb2775379c92c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34599 } } peers { permanent_uuid: "648b7f0c387e4ea98cd4628a1fea8e06" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 44305 } } peers { permanent_uuid: "71cd3a3ec6e64128ae9ffba06c9b6f6c" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 43389 } } }
I20260812 14:49:43.320742 7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:49:43.320775 7320 mvcc.cc:205] Tried to move back new op lower bound from 7317693166865256448 to 7317693166858350592. Current Snapshot: MvccSnapshot[applied={T|T < 7317693166865256448}]
I20260812 14:49:43.321079 7319 mvcc.cc:205] Tried to move back new op lower bound from 7317693166865256448 to 7317693166858350592. Current Snapshot: MvccSnapshot[applied={T|T < 7317693166865256448}]
I20260812 14:49:43.322009 7316 catalog_manager.cc:1347] Generated new cluster ID: 6243eb2c1d024410b8964e5bfac57afe
I20260812 14:49:43.322077 7316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:49:43.340756 7316 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:49:43.341619 7316 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:43.348043 7316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 648b7f0c387e4ea98cd4628a1fea8e06: Generated new TSK 0
I20260812 14:49:43.348259 7316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:49:44.087177 7256 catalog_manager.cc:1259] Loaded cluster ID: 6243eb2c1d024410b8964e5bfac57afe
I20260812 14:49:44.087298 7256 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: loading cluster ID for follower catalog manager: success
I20260812 14:49:44.088137 7256 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: acquiring CA information for follower catalog manager: success
I20260812 14:49:44.088466 7256 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71cd3a3ec6e64128ae9ffba06c9b6f6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:49:44.164389 7312 catalog_manager.cc:1259] Loaded cluster ID: 6243eb2c1d024410b8964e5bfac57afe
I20260812 14:49:44.164479 7312 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: loading cluster ID for follower catalog manager: success
I20260812 14:49:44.165413 7312 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: acquiring CA information for follower catalog manager: success
I20260812 14:49:44.165704 7312 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89413283058f4cc8a41fb2775379c92c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:49:44.416811 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 6803
I20260812 14:49:44.431525 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 6952
I20260812 14:49:44.450292 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 7092
2026-08-12T14:49:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3370 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:49:44 dist-test-slave-8hhm krb5kdc[7348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:49:44 dist-test-slave-8hhm krb5kdc[7348](info): set up 2 sockets
Aug 12 14:49:44 dist-test-slave-8hhm krb5kdc[7348](info): commencing operation
krb5kdc: starting...
W20260812 14:49:45.771432 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:49:45 dist-test-slave-8hhm krb5kdc[7348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:49:45.900976 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:38217
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:38217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:45.985357 7372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:45.985610 7372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:45.985648 7372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:45.986829 7372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:45.986905 7372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:45.986930 7372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:45.986950 7372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:45.986981 7372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:45.988505 7372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:38217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:38217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:45.988814 7372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:45.989059 7372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:45.993993 7386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:45.994024 7390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:45.994029 7384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:45.994315 7372 server_base.cc:1034] running on GCE node
I20260812 14:49:45.994557 7372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:45.994855 7372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:45.996024 7372 hybrid_clock.cc:648] HybridClock initialized: now 1786546185995998 us; error 62 us; skew 500 ppm
Aug 12 14:49:45 dist-test-slave-8hhm krb5kdc[7348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546185, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:45.998631 7372 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:45.999100 7372 webserver.cc:460] Webserver started at http://127.13.146.190:41163/ using document root <none> and password file <none>
I20260812 14:49:45.999346 7372 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:45.999415 7372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:45.999573 7372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:46.000658 7372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "810bd95e18dc499e94ddfb8ed1e6e700"
format_stamp: "Formatted at 2026-08-12 14:49:45 on dist-test-slave-8hhm"
server_key: "cd093986392a54bce2d8d1b34744eae1"
server_key_iv: "ea3ac410275e368a48989ee6a1aba03f"
server_key_version: "encryptionkey@0"
I20260812 14:49:46.001053 7372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "810bd95e18dc499e94ddfb8ed1e6e700"
format_stamp: "Formatted at 2026-08-12 14:49:45 on dist-test-slave-8hhm"
server_key: "cd093986392a54bce2d8d1b34744eae1"
server_key_iv: "ea3ac410275e368a48989ee6a1aba03f"
server_key_version: "encryptionkey@0"
I20260812 14:49:46.002513 7372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:46.003300 7410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:46.003474 7372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:46.003589 7372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "810bd95e18dc499e94ddfb8ed1e6e700"
format_stamp: "Formatted at 2026-08-12 14:49:45 on dist-test-slave-8hhm"
server_key: "cd093986392a54bce2d8d1b34744eae1"
server_key_iv: "ea3ac410275e368a48989ee6a1aba03f"
server_key_version: "encryptionkey@0"
I20260812 14:49:46.003669 7372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:46.013768 7372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:46.015664 7372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:46.015827 7372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:46.020071 7372 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:38217
I20260812 14:49:46.020126 7511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:38217 every 8 connection(s)
I20260812 14:49:46.020509 7372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:46.021481 7513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:46.023800 7513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: Bootstrap starting.
I20260812 14:49:46.024410 7513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:46.024717 7513 log.cc:824] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:46.025547 7513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: No bootstrap required, opened a new log
I20260812 14:49:46.026782 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 7372
I20260812 14:49:46.026867 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:49:46.027064 13898 external_mini_cluster.cc:1428] Setting key e72313ac13007e96c8f2fb996d6ec0cb
I20260812 14:49:46.027405 7513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } }
I20260812 14:49:46.027643 7513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:46.027704 7513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810bd95e18dc499e94ddfb8ed1e6e700, State: Initialized, Role: FOLLOWER
I20260812 14:49:46.027837 7513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } }
I20260812 14:49:46.027937 7513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:46.027985 7513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:46.028039 7513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:46.028893 7513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } }
I20260812 14:49:46.029019 7513 leader_election.cc:302] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 810bd95e18dc499e94ddfb8ed1e6e700; no voters:
I20260812 14:49:46.029199 7513 leader_election.cc:288] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:49:46.029268 7526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:46.029448 7526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [term 1 LEADER]: Becoming Leader. State: Replica: 810bd95e18dc499e94ddfb8ed1e6e700, State: Running, Role: LEADER
I20260812 14:49:46.029500 7513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:46.029563 7526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } }
I20260812 14:49:46.029918 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 810bd95e18dc499e94ddfb8ed1e6e700. Latest consensus state: current_term: 1 leader_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } } }
I20260812 14:49:46.030011 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:46.030193 7527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810bd95e18dc499e94ddfb8ed1e6e700" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 38217 } } }
I20260812 14:49:46.030471 7527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700 [sys.catalog]: This master's current role is: LEADER
W20260812 14:49:46.031836 7554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:49:46.031893 7554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:49:46.032550 7537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:46.032784 7537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:46.034444 7537 catalog_manager.cc:1347] Generated new cluster ID: b62fe1a6732d4e439f6b4bbb8b90dece
I20260812 14:49:46.034495 7537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:49:46 dist-test-slave-8hhm krb5kdc[7348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:46.038995 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:46.027611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.13.146.190:38217)
0812 14:49:46.027826 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:46.027835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:46.027865 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:49:46.027988 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:46.027995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:46.028051 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:46.028186 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:46.028805 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:46.030542 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:46.031634 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:46.032231 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:46.032351 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:46.035147 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:46.035151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:46.035152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:46.035164 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:46.037277 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:46.037664 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:46.037667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:46.037667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:46.037796 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:46.038014 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:46.038015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:46.038015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:46.038393 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:46.038594 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:46.038672 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:49:46.038805 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260812 14:49:46.040045 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:49:46.046926 7537 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:49:46.047631 7537 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:46.050588 7537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 810bd95e18dc499e94ddfb8ed1e6e700: Generated new TSK 0
I20260812 14:49:46.050756 7537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:49:46.094115 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 7372
2026-08-12T14:49:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1648 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:49:46 dist-test-slave-8hhm krb5kdc[7561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:49:46 dist-test-slave-8hhm krb5kdc[7561](info): set up 2 sockets
Aug 12 14:49:46 dist-test-slave-8hhm krb5kdc[7561](info): commencing operation
krb5kdc: starting...
W20260812 14:49:47.452564 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:49:47 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:49:47.580477 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:39727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:47.665470 7594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:47.665741 7594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:47.665788 7594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:47.666946 7594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:47.667019 7594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:47.667044 7594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:47.667074 7594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:47.667105 7594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:47.668603 7594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:39727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:47.668902 7594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:47.669138 7594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:47.674194 7615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:47.674208 7610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:47.674194 7606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:47.674535 7594 server_base.cc:1034] running on GCE node
I20260812 14:49:47.674832 7594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:47.675156 7594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:47.676343 7594 hybrid_clock.cc:648] HybridClock initialized: now 1786546187676318 us; error 40 us; skew 500 ppm
Aug 12 14:49:47 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:47.691288 7594 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:47.691795 7594 webserver.cc:460] Webserver started at http://127.13.146.190:35505/ using document root <none> and password file <none>
I20260812 14:49:47.692052 7594 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:47.692126 7594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:47.692281 7594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:47.693496 7594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "be67d8a0701941ab96d0350161b7e029"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "611d870246d881962428dcc8f5548a2b"
server_key_iv: "50a577fee589a019b92aa793894236c8"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.693874 7594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "be67d8a0701941ab96d0350161b7e029"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "611d870246d881962428dcc8f5548a2b"
server_key_iv: "50a577fee589a019b92aa793894236c8"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.695284 7594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:49:47.696077 7633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:47.696290 7594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:47.696410 7594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "be67d8a0701941ab96d0350161b7e029"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "611d870246d881962428dcc8f5548a2b"
server_key_iv: "50a577fee589a019b92aa793894236c8"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.696491 7594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:47.709782 7594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:47.711918 7594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:47.712103 7594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:47.716542 7594 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:39727
I20260812 14:49:47.716594 7721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:39727 every 8 connection(s)
I20260812 14:49:47.716979 7594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:47.718060 7726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:47.720364 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap starting.
I20260812 14:49:47.720930 7726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:47.721168 7726 log.cc:824] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:47.721975 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: No bootstrap required, opened a new log
I20260812 14:49:47.723567 7726 raft_consensus.cc:348] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:47.723716 7726 raft_consensus.cc:374] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:47.723740 7726 raft_consensus.cc:729] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Initialized, Role: FOLLOWER
I20260812 14:49:47.723882 7726 consensus_queue.cc:260] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:47.723968 7726 raft_consensus.cc:388] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:47.723992 7726 raft_consensus.cc:482] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:47.724022 7726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:47.724812 7726 raft_consensus.cc:504] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:47.724902 7726 leader_election.cc:302] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be67d8a0701941ab96d0350161b7e029; no voters:
I20260812 14:49:47.725025 7726 leader_election.cc:288] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:49:47.725113 7731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:47.725319 7731 raft_consensus.cc:686] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 LEADER]: Becoming Leader. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Running, Role: LEADER
I20260812 14:49:47.725382 7726 sys_catalog.cc:564] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:47.725428 7731 consensus_queue.cc:237] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:47.725831 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:47.725939 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:47.725842 7733 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be67d8a0701941ab96d0350161b7e029. Latest consensus state: current_term: 1 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:47.726025 7733 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:47.726193 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 7594
I20260812 14:49:47.726315 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:49:47.726573 13898 external_mini_cluster.cc:1428] Setting key 4b37ad286cf2abbc0e02f6e2df7ea001
I20260812 14:49:47.726702 7750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:47.727368 7750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:47.728916 7750 catalog_manager.cc:1347] Generated new cluster ID: c40c364d567f4dc3ad041163a9ac7866
I20260812 14:49:47.728974 7750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:49:47.733860 7750 catalog_manager.cc:1370] Generated new certificate authority record
Aug 12 14:49:47 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:47.734512 7750 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:47.738591 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:47.727056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.13.146.190:39727)
0812 14:49:47.727229 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:47.727247 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:47.727285 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:49:47.727419 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:47.727427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:47.727498 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:47.727656 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:47.728452 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.729886 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.730974 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.731556 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.731647 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:47.734776 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:47.734780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:47.734780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:47.734791 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:47.736868 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:47.737330 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:47.737333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:47.737333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:47.737452 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:47.737676 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:47.737677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:47.737677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:47.737935 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:47.738095 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:47.738206 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:49:47.738349 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260812 14:49:47.739462 7750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Generated new TSK 0
I20260812 14:49:47.739619 7750 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:49:47.772387 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:47.859206 7777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:47.859409 7777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:47.859433 7777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:47.860512 7777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:47.860545 7777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:47.860571 7777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:49:47.861979 7777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:47.862167 7777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:47.862353 7777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:47.867406 7783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:47.867436 7792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:47.867690 7777 server_base.cc:1034] running on GCE node
W20260812 14:49:47.867478 7786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:47.867923 7777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:47.868144 7777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:47.869287 7777 hybrid_clock.cc:648] HybridClock initialized: now 1786546187869264 us; error 46 us; skew 500 ppm
Aug 12 14:49:47 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:47.871868 7777 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:49:47.872331 7777 webserver.cc:460] Webserver started at http://127.13.146.129:33535/ using document root <none> and password file <none>
I20260812 14:49:47.872584 7777 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:47.872633 7777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:47.872798 7777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:47.873914 7777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "b3fe2172a5522dbd1ab67f1fe53a05cc"
server_key_iv: "c8dde3adab959c46fbafc993c1f78f35"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.874284 7777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "b3fe2172a5522dbd1ab67f1fe53a05cc"
server_key_iv: "c8dde3adab959c46fbafc993c1f78f35"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.875712 7777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:49:47.876451 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:47.876633 7777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:49:47.876739 7777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "b3fe2172a5522dbd1ab67f1fe53a05cc"
server_key_iv: "c8dde3adab959c46fbafc993c1f78f35"
server_key_version: "encryptionkey@0"
I20260812 14:49:47.876827 7777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:47.891562 7777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:47.893436 7777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:47.893610 7777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:47.893852 7777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:47.894183 7777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:47.894240 7777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:47.894273 7777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:47.894300 7777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:47.900728 7777 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:36819
I20260812 14:49:47.900777 8013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:36819 every 8 connection(s)
I20260812 14:49:47.901087 7777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:49:47 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546187, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:47.908169 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 7777
I20260812 14:49:47.908322 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:49:47.908555 13898 external_mini_cluster.cc:1428] Setting key 99d40b588f780797309c5535cf102fe6
I20260812 14:49:47.910660 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:47.901835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:44277)
0812 14:49:47.901996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:47.902038 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:47.902532 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:47.902741 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:47.902745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:47.903316 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:47.903513 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:47.903518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.904210 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.904210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:47.904999 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:47.905001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.905430 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.905430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:47.905531 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:47.905976 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:47.905984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:47.907369 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:47.909291 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:47.909300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:47.909303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:47.909538 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:47.909767 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:47.909768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:47.909768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:47.909870 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:47.910178 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:47.910183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:47.910294 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 14:49:47.910378 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:49:47.910499 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260812 14:49:47.910696 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:47.901883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:44277 (local address 127.13.146.190:39727)
0812 14:49:47.901936 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:47.901938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:47.902131 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:49:47.902619 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:47.902620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:47.902630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:47.902689 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:47.903599 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.904099 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.905086 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:47.905356 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:47.905413 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:47.907459 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:47.907461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:47.907461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:47.907468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:47.909175 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:47.909616 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:47.909617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:47.909617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:47.909686 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:47.909934 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:47.909935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:47.909936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:47.910094 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:47.910168 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:47.910449 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 14:49:47.910561 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:49:47.911531 8015 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:47.911649 8015 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:47.911865 8015 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
I20260812 14:49:47.912432 7670 ts_manager.cc:194] Registered new tserver with Master: 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:47.913378 7670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:44277
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:49:47.936751 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:48.020411 8023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:48.020619 8023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:48.020643 8023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:48.021929 8023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:48.021971 8023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:48.021999 8023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:49:48.023576 8023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:48.023844 8023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:48.024096 8023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:48.029567 8040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.029544 8032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.029539 8030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:48.029803 8023 server_base.cc:1034] running on GCE node
I20260812 14:49:48.029985 8023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:48.030180 8023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:48.031318 8023 hybrid_clock.cc:648] HybridClock initialized: now 1786546188031306 us; error 27 us; skew 500 ppm
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:48.033993 8023 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:49:48.034384 8023 webserver.cc:460] Webserver started at http://127.13.146.130:44315/ using document root <none> and password file <none>
I20260812 14:49:48.034590 8023 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:48.034633 8023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:48.034723 8023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:48.035820 8023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "1d84c901b5ff4820a6ffe0c7064ce91b"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "d7201002e82ab1caf99af92a809f8094"
server_key_iv: "d7dd09f6bcb1a3e2bf4e4c957841cb86"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.036187 8023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "1d84c901b5ff4820a6ffe0c7064ce91b"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "d7201002e82ab1caf99af92a809f8094"
server_key_iv: "d7dd09f6bcb1a3e2bf4e4c957841cb86"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.037669 8023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:49:48.038430 8053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.038628 8023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:48.038719 8023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "1d84c901b5ff4820a6ffe0c7064ce91b"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "d7201002e82ab1caf99af92a809f8094"
server_key_iv: "d7dd09f6bcb1a3e2bf4e4c957841cb86"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.038776 8023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:48.071898 8023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:48.074038 8023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:48.074154 8023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:48.074368 8023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:48.074712 8023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:48.074750 8023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.074774 8023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:48.074786 8023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.080776 8023 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:36119
I20260812 14:49:48.080827 8253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:36119 every 8 connection(s)
I20260812 14:49:48.081156 8023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:49:48.082540 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 8023
I20260812 14:49:48.082652 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:49:48.082862 13898 external_mini_cluster.cc:1428] Setting key fd0a3a28c2009be0d3b0d300aab5aabe
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:48.092000 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.081970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:45489 (local address 127.13.146.190:39727)
0812 14:49:48.082018 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.082021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.082233 (+ 212us) server_negotiation.cc:407] Connection header received
0812 14:49:48.082965 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.082966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.082979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.083120 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.084473 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.084946 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.086213 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.086531 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.086612 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.088740 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.088741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.088742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.088749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.090605 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.090996 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.090998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.090998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.091079 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.091325 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.091326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.091326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.091486 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.091562 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.091815 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.091902 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:49:48.092037 8267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.081925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:45489)
0812 14:49:48.082089 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.082133 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.082845 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.083250 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.083254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.084073 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:48.084376 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.084383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.085043 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.085044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.086112 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.086115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.086623 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.086624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.086731 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.087297 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:48.087305 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:48.088648 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:48.090690 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.090696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.090698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.090915 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.091149 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.091150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.091150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.091255 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.091556 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:48.091561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:48.091670 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.091762 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.091877 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260812 14:49:48.092610 8259 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:48.092725 8259 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:48.092924 8259 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:49:48.093382 7670 ts_manager.cc:194] Registered new tserver with Master: 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
I20260812 14:49:48.093778 7670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:45489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:49:48.112269 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:48.196590 8272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:48.196825 8272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:48.196864 8272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:48.198036 8272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:48.198102 8272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:48.198146 8272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:49:48.199604 8272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:48.199860 8272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:48.200143 8272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:48.205129 8288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.205181 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.205200 8279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:48.205500 8272 server_base.cc:1034] running on GCE node
I20260812 14:49:48.205713 8272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:48.205958 8272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:48.207125 8272 hybrid_clock.cc:648] HybridClock initialized: now 1786546188207107 us; error 38 us; skew 500 ppm
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:48.209698 8272 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:49:48.210201 8272 webserver.cc:460] Webserver started at http://127.13.146.131:33947/ using document root <none> and password file <none>
I20260812 14:49:48.210436 8272 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:48.210505 8272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:48.210644 8272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:49:48.211701 8272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "196e826eda1d4d2eae3a0efb67e561c3"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "c081008f55c194149a26498792e3e1b4"
server_key_iv: "0c1af8bb5e3f0a234a4aaf26de082b1d"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.212111 8272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "196e826eda1d4d2eae3a0efb67e561c3"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "c081008f55c194149a26498792e3e1b4"
server_key_iv: "0c1af8bb5e3f0a234a4aaf26de082b1d"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.213472 8272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:49:48.214231 8296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.214419 8272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:49:48.214526 8272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "196e826eda1d4d2eae3a0efb67e561c3"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "c081008f55c194149a26498792e3e1b4"
server_key_iv: "0c1af8bb5e3f0a234a4aaf26de082b1d"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.214600 8272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:48.221843 8272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:48.224148 8272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:48.224318 8272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:48.224550 8272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:48.224866 8272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:48.224925 8272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.224965 8272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:48.224993 8272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.231479 8272 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:46671
I20260812 14:49:48.231525 8521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:46671 every 8 connection(s)
I20260812 14:49:48.231853 8272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:49:48.237594 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 8272
I20260812 14:49:48.237728 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:49:48.237937 13898 external_mini_cluster.cc:1428] Setting key eaab2aa57febbe3eb00c63adb8c9cb9e
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:48.241691 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.232555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51893 (local address 127.13.146.190:39727)
0812 14:49:48.232600 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.232602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.232830 (+ 228us) server_negotiation.cc:407] Connection header received
0812 14:49:48.233465 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.233466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.233479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.233541 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.234478 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.234996 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.236098 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.236296 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.236385 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.239018 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.239019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.239019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.239045 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.240388 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.240760 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.240761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.240761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.240811 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.241024 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.241024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.241025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.241151 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.241203 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.241534 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.241619 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:49:48.241884 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.232530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:51893)
0812 14:49:48.232711 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.232745 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.233335 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.233610 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.233614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.234184 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:48.234365 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.234370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.235098 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.235099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.235991 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.235993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.236395 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.236396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.236530 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.237091 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:48.237099 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:48.238933 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:48.240463 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.240469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.240471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.240685 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.240871 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.240872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.240872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.240970 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.241208 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:48.241254 (+ 46us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:48.241384 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.241470 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.241640 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260812 14:49:48.242368 8524 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:48.242461 8524 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:48.242597 8524 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
I20260812 14:49:48.242938 7670 ts_manager.cc:194] Registered new tserver with Master: 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671)
I20260812 14:49:48.243324 7670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:51893
I20260812 14:49:48.249931 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:49:48.257452 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.251302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.13.146.190:39727)
0812 14:49:48.251340 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.251342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.251464 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:49:48.251540 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.251540 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.251551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.251630 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.252232 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.252706 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.253508 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.253705 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.253782 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.254908 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.254909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.254910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.254918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.256098 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.256499 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.256500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.256500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.256578 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.256784 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.256785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.256786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.256943 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.257051 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.257264 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.257349 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:49:48.258996 7670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:49:48.260612 7670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:49:48.267484 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:36119 (local address 127.0.0.1:51812)
0812 14:49:48.263434 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.263481 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.263597 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.263979 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.263982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.263984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:48.264067 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.264069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.264975 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.264976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.266412 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.266415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267142 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.267230 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267244 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.267341 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.267400 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260812 14:49:48.267751 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39176 (local address 127.13.146.129:36819)
0812 14:49:48.263779 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.263783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.263793 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:49:48.263830 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.263832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.263847 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.263931 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:48.264184 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.265324 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.266497 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267222 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267306 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267366 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.267455 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.267496 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":100,"threads_started":1}
I20260812 14:49:48.267761 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51812 (local address 127.13.146.130:36119)
0812 14:49:48.263467 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.263472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.263556 (+ 84us) server_negotiation.cc:407] Connection header received
0812 14:49:48.263759 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.263762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.263780 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.263856 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:48.264158 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.264869 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.266519 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267046 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267141 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267202 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.267667 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.267701 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260812 14:49:48.267881 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.13.146.131:46671)
0812 14:49:48.263724 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.263728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.263738 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:49:48.263779 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.263782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.263798 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.263875 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:48.264163 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.265322 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.266626 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267291 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267351 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267395 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.267733 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.267784 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":83,"threads_started":1}
I20260812 14:49:48.267558 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.0.0.1:39176)
0812 14:49:48.263518 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.263527 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.263568 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.264003 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.264004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.264006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:48.264101 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.264103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.265453 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.265454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.266412 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.266414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267312 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.267369 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267372 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.267424 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.267457 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":42,"thread_start_us":75,"threads_started":1}
I20260812 14:49:48.267661 8545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.263169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:46671 (local address 127.0.0.1:36772)
0812 14:49:48.263481 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.263492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.263561 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.263980 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.263982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.263985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:48.264075 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.264077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.265447 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.265448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.266551 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.266553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.267436 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.267437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.267501 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.267505 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.267559 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.267597 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":39,"thread_start_us":100,"threads_started":1}
I20260812 14:49:48.268671 7885 tablet_service.cc:1467] Processing CreateTablet for tablet 37a23a8b888d413580e2faa1549ad57b (DEFAULT_TABLE table=test-table [id=f1e697a64de44ea0a3a9982c18578d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:48.268659 8146 tablet_service.cc:1467] Processing CreateTablet for tablet 37a23a8b888d413580e2faa1549ad57b (DEFAULT_TABLE table=test-table [id=f1e697a64de44ea0a3a9982c18578d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:48.269038 7885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a23a8b888d413580e2faa1549ad57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:48.269050 8146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a23a8b888d413580e2faa1549ad57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:48.269673 8380 tablet_service.cc:1467] Processing CreateTablet for tablet 37a23a8b888d413580e2faa1549ad57b (DEFAULT_TABLE table=test-table [id=f1e697a64de44ea0a3a9982c18578d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:48.270020 8380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a23a8b888d413580e2faa1549ad57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:48.271946 8562 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Bootstrap starting.
I20260812 14:49:48.272646 8565 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap starting.
I20260812 14:49:48.272706 8562 tablet_bootstrap.cc:654] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:48.273396 8565 tablet_bootstrap.cc:654] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:48.273448 8562 log.cc:824] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:48.273676 8566 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Bootstrap starting.
I20260812 14:49:48.273931 8565 log.cc:824] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:48.274390 8566 tablet_bootstrap.cc:654] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:48.274658 8562 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: No bootstrap required, opened a new log
I20260812 14:49:48.274739 8562 ts_tablet_manager.cc:1397] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:49:48.274799 8565 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: No bootstrap required, opened a new log
I20260812 14:49:48.274875 8565 ts_tablet_manager.cc:1397] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:49:48.274969 8566 log.cc:824] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:48.275799 8566 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: No bootstrap required, opened a new log
I20260812 14:49:48.275882 8566 ts_tablet_manager.cc:1397] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:49:48.276628 8565 raft_consensus.cc:348] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.276799 8565 raft_consensus.cc:374] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:48.276845 8565 raft_consensus.cc:729] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dff3e56e8ff4a5b978aaf628f463c85, State: Initialized, Role: FOLLOWER
I20260812 14:49:48.276876 8562 raft_consensus.cc:348] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.276984 8565 consensus_queue.cc:260] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.277045 8562 raft_consensus.cc:374] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:48.277100 8562 raft_consensus.cc:729] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d84c901b5ff4820a6ffe0c7064ce91b, State: Initialized, Role: FOLLOWER
I20260812 14:49:48.277206 8562 consensus_queue.cc:260] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.277287 8565 ts_tablet_manager.cc:1428] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:49:48.277426 8015 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
I20260812 14:49:48.277519 8562 ts_tablet_manager.cc:1428] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:49:48.277609 8259 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
I20260812 14:49:48.278244 8566 raft_consensus.cc:348] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.278548 8566 raft_consensus.cc:374] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:48.278632 8566 raft_consensus.cc:729] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196e826eda1d4d2eae3a0efb67e561c3, State: Initialized, Role: FOLLOWER
I20260812 14:49:48.278748 8566 consensus_queue.cc:260] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.279047 8566 ts_tablet_manager.cc:1428] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:49:48.279114 8524 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
W20260812 14:49:48.331835 8265 tablet.cc:2369] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:49:48.394284 8574 raft_consensus.cc:482] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:48.394416 8574 raft_consensus.cc:504] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.394874 8574 leader_election.cc:288] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:48.398775 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.395206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:43651 (local address 127.13.146.129:36819)
0812 14:49:48.395265 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.395268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.395325 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:49:48.395404 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.395405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.395418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.395473 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:48.395633 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.396305 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.397816 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.398395 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.398480 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.398486 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.398672 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.398718 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:49:48.398770 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.395148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42953 (local address 127.13.146.131:46671)
0812 14:49:48.395194 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.395197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.395242 (+ 45us) server_negotiation.cc:407] Connection header received
0812 14:49:48.395279 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.395280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.395287 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.395334 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:48.395541 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.396283 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.397817 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.398395 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.398477 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.398486 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.398662 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.398701 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:49:48.398780 8267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.395066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:46671 (local address 127.13.146.130:42953)
0812 14:49:48.395127 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.395140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.395187 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.395357 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.395358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.395360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:48.395442 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.395444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.396406 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.396407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.397729 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.397732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.398485 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.398486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.398588 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.398591 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.398644 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.398680 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":70}
I20260812 14:49:48.398785 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.395090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.13.146.130:43651)
0812 14:49:48.395273 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.395281 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.395331 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.395501 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.395502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.395503 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:48.395561 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.395562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.396406 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.396407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.397729 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.397732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.398485 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.398486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.398565 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.398572 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.398633 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.398679 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":59,"thread_start_us":122,"threads_started":1}
I20260812 14:49:48.399245 8428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a23a8b888d413580e2faa1549ad57b" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196e826eda1d4d2eae3a0efb67e561c3" is_pre_election: true
I20260812 14:49:48.399286 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a23a8b888d413580e2faa1549ad57b" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" is_pre_election: true
I20260812 14:49:48.399415 8428 raft_consensus.cc:2454] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 0.
I20260812 14:49:48.399489 7915 raft_consensus.cc:2454] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 0.
I20260812 14:49:48.399715 8056 leader_election.cc:302] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b; no voters:
I20260812 14:49:48.399843 8574 raft_consensus.cc:2781] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:49:48.399923 8574 raft_consensus.cc:482] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:49:48.399958 8574 raft_consensus.cc:3037] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:48.400775 8574 raft_consensus.cc:504] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.400925 8574 leader_election.cc:288] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 election: Requested vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:48.401093 8428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a23a8b888d413580e2faa1549ad57b" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196e826eda1d4d2eae3a0efb67e561c3"
I20260812 14:49:48.401116 7915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a23a8b888d413580e2faa1549ad57b" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
I20260812 14:49:48.401175 8428 raft_consensus.cc:3037] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:48.401199 7915 raft_consensus.cc:3037] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Advancing to term 1
W20260812 14:49:48.401859 8016 tablet.cc:2369] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:49:48.402120 7915 raft_consensus.cc:2454] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 1.
I20260812 14:49:48.402244 8428 raft_consensus.cc:2454] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 1.
I20260812 14:49:48.402298 8058 leader_election.cc:302] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d84c901b5ff4820a6ffe0c7064ce91b, 2dff3e56e8ff4a5b978aaf628f463c85; no voters:
I20260812 14:49:48.402377 8574 raft_consensus.cc:2781] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:48.402583 8574 raft_consensus.cc:686] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 LEADER]: Becoming Leader. State: Replica: 1d84c901b5ff4820a6ffe0c7064ce91b, State: Running, Role: LEADER
I20260812 14:49:48.402704 8574 consensus_queue.cc:237] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.403591 7669 catalog_manager.cc:5495] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130). New cstate: current_term: 1 leader_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } health_report { overall_health: UNKNOWN } } }
I20260812 14:49:48.417076 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.414596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.13.146.130:36119)
0812 14:49:48.414657 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.414659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.414668 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:49:48.414709 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.414710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.414738 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.414818 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:49:48.414941 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.415427 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.416238 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.416438 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.416518 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.416695 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:49:48.416810 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:49:48.416902 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.416942 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.416963 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:49:48.419194 8428 raft_consensus.cc:1261] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Refusing update from remote peer 1d84c901b5ff4820a6ffe0c7064ce91b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:48.419227 7915 raft_consensus.cc:1261] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Refusing update from remote peer 1d84c901b5ff4820a6ffe0c7064ce91b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:48.419550 8580 consensus_queue.cc:1035] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Connected to new peer: Peer: permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:48.419694 8574 consensus_queue.cc:1035] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:48.422428 8585 mvcc.cc:205] Tried to move back new op lower bound from 7317693187761778688 to 7317693187698098176. Current Snapshot: MvccSnapshot[applied={T|T < 7317693187761778688}]
I20260812 14:49:48.422539 8586 mvcc.cc:205] Tried to move back new op lower bound from 7317693187761778688 to 7317693187698098176. Current Snapshot: MvccSnapshot[applied={T|T < 7317693187761778688}]
I20260812 14:49:48.422724 8587 mvcc.cc:205] Tried to move back new op lower bound from 7317693187761778688 to 7317693187698098176. Current Snapshot: MvccSnapshot[applied={T|T < 7317693187761778688}]
I20260812 14:49:48.424455 7669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:48.424542 7669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:48.425398 7669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Sending DeleteTablet for 3 replicas of tablet 37a23a8b888d413580e2faa1549ad57b
I20260812 14:49:48.425758 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 7777
I20260812 14:49:48.425776 8380 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a23a8b888d413580e2faa1549ad57b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:48 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:36772
I20260812 14:49:48.425791 8146 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a23a8b888d413580e2faa1549ad57b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:48 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:51812
I20260812 14:49:48.437067 8599 tablet_replica.cc:330] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: stopping tablet replica
I20260812 14:49:48.437361 8602 tablet_replica.cc:330] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: stopping tablet replica
I20260812 14:49:48.437487 8602 raft_consensus.cc:2229] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:49:48.437613 8602 raft_consensus.cc:2258] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:48.437745 8599 raft_consensus.cc:2229] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:49:48.437886 8599 raft_consensus.cc:2258] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:48.438462 8602 ts_tablet_manager.cc:1905] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:48.440655 8602 ts_tablet_manager.cc:1918] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:48.440748 8602 log.cc:1196] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/37a23a8b888d413580e2faa1549ad57b
I20260812 14:49:48.441073 8602 ts_tablet_manager.cc:1939] T 37a23a8b888d413580e2faa1549ad57b P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting consensus metadata
I20260812 14:49:48.441787 7643 catalog_manager.cc:4841] TS 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): tablet 37a23a8b888d413580e2faa1549ad57b (table test-table [id=f1e697a64de44ea0a3a9982c18578d19]) successfully deleted
I20260812 14:49:48.441874 8599 ts_tablet_manager.cc:1905] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:48.444332 8599 ts_tablet_manager.cc:1918] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:48.444425 8599 log.cc:1196] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/37a23a8b888d413580e2faa1549ad57b
W20260812 14:49:48.444571 7643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:36819 (error 108)
I20260812 14:49:48.444824 8599 ts_tablet_manager.cc:1939] T 37a23a8b888d413580e2faa1549ad57b P 196e826eda1d4d2eae3a0efb67e561c3: Deleting consensus metadata
I20260812 14:49:48.445279 8545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.445107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.0.0.1:39178)
0812 14:49:48.445174 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.445239 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 14:49:48.445329 7636 catalog_manager.cc:4841] TS 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): tablet 37a23a8b888d413580e2faa1549ad57b (table test-table [id=f1e697a64de44ea0a3a9982c18578d19]) successfully deleted
W20260812 14:49:48.445407 7643 catalog_manager.cc:4584] TS 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37a23a8b888d413580e2faa1549ad57b: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36819: connect: Connection refused (error 111)
I20260812 14:49:48.445729 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 8023
I20260812 14:49:48.462710 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 8272
I20260812 14:49:48.478037 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 7594
I20260812 14:49:48.491338 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--webserver_interface=127.13.146.190
--webserver_port=35505
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:39727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:48.575165 8604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:48.575397 8604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:48.575420 8604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:48.576646 8604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:48.576695 8604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:48.576709 8604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:48.576721 8604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:48.576777 8604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:48.578329 8604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:39727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:48.578616 8604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:48.578881 8604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:48.588440 8619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.588441 8614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.588519 8615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:48.588608 8604 server_base.cc:1034] running on GCE node
I20260812 14:49:48.588943 8604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:48.589339 8604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:48.590497 8604 hybrid_clock.cc:648] HybridClock initialized: now 1786546188590479 us; error 35 us; skew 500 ppm
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:48.593024 8604 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:48.593524 8604 webserver.cc:460] Webserver started at http://127.13.146.190:35505/ using document root <none> and password file <none>
I20260812 14:49:48.593786 8604 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:48.593870 8604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:48.595228 8604 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:48.595925 8630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.596112 8604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:49:48.596228 8604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "be67d8a0701941ab96d0350161b7e029"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "611d870246d881962428dcc8f5548a2b"
server_key_iv: "50a577fee589a019b92aa793894236c8"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.596678 8604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:48.622136 8604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:48.624158 8604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:48.624338 8604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:48.628830 8604 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:39727
I20260812 14:49:48.628866 8717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:39727 every 8 connection(s)
I20260812 14:49:48.629331 8604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:48.631039 8718 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:49:48.631801 8718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap starting.
I20260812 14:49:48.637106 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 8604
I20260812 14:49:48.637456 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:36819
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=33535
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:48.641683 8718 log.cc:824] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:48.643672 8718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:49:48.644003 8718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap complete.
I20260812 14:49:48.646512 8718 raft_consensus.cc:348] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:48.646775 8718 raft_consensus.cc:729] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Initialized, Role: FOLLOWER
I20260812 14:49:48.646988 8718 consensus_queue.cc:260] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:48.647089 8718 raft_consensus.cc:388] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:48.647150 8718 raft_consensus.cc:482] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:48.647199 8718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:49:48.648299 8718 raft_consensus.cc:504] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:48.648428 8718 leader_election.cc:302] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be67d8a0701941ab96d0350161b7e029; no voters:
I20260812 14:49:48.648617 8718 leader_election.cc:288] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:49:48.648686 8724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:49:48.648844 8724 raft_consensus.cc:686] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 LEADER]: Becoming Leader. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Running, Role: LEADER
I20260812 14:49:48.648905 8718 sys_catalog.cc:564] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:48.648941 8724 consensus_queue.cc:237] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:48.649251 8724 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be67d8a0701941ab96d0350161b7e029. Latest consensus state: current_term: 2 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:48.649341 8724 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:48.649252 8726 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:48.649524 8737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:48.649544 8726 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:48.650352 8737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:48.650456 8737 catalog_manager.cc:1259] Loaded cluster ID: c40c364d567f4dc3ad041163a9ac7866
I20260812 14:49:48.650650 8737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:49:48.651160 8737 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 14:49:48.656230 8737 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 14:49:48.659332 8737 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:49:48.659519 8737 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Loaded TSK: 0
I20260812 14:49:48.659741 8737 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:49:48.727666 8722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:48.727898 8722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:48.727924 8722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:48.729153 8722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:48.729195 8722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:48.729271 8722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:49:48.730777 8722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:36819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=33535
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:48.731011 8722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:48.731281 8722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:48.737524 8791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.737525 8795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.737576 8789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:48.737874 8722 server_base.cc:1034] running on GCE node
I20260812 14:49:48.738111 8722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:48.738469 8722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:48.739682 8722 hybrid_clock.cc:648] HybridClock initialized: now 1786546188739673 us; error 35 us; skew 500 ppm
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:48.742901 8722 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:49:48.743544 8722 webserver.cc:460] Webserver started at http://127.13.146.129:33535/ using document root <none> and password file <none>
I20260812 14:49:48.743800 8722 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:48.743876 8722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:48.745801 8722 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:49:48.746703 8808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.746953 8722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:48.747057 8722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "b3fe2172a5522dbd1ab67f1fe53a05cc"
server_key_iv: "c8dde3adab959c46fbafc993c1f78f35"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.747538 8722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:48.765419 8722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:48.767926 8722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:48.768097 8722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:48.768391 8722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:48.768872 8816 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:49:48.770439 8722 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:49:48.770505 8722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:49:48.770539 8722 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:49:48.771384 8722 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:49:48.771427 8722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:49:48.771509 8816 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap starting.
I20260812 14:49:48.778563 8722 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:36819
I20260812 14:49:48.778800 9010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:36819 every 8 connection(s)
I20260812 14:49:48.779021 8722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:49:48.782894 8816 log.cc:824] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:48.783946 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 8722
I20260812 14:49:48.784266 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:36119
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=44315
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:48.785832 8816 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:49:48.786170 8816 tablet_bootstrap.cc:492] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap complete.
I20260812 14:49:48.787523 8816 ts_tablet_manager.cc:1397] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:48.790123 8816 raft_consensus.cc:348] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.790548 8816 raft_consensus.cc:729] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dff3e56e8ff4a5b978aaf628f463c85, State: Initialized, Role: FOLLOWER
I20260812 14:49:48.790768 8816 consensus_queue.cc:260] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:48.791114 8816 ts_tablet_manager.cc:1428] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260812 14:49:48.794164 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.779893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:34307 (local address 127.13.146.190:39727)
0812 14:49:48.781349 (+ 1456us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.781358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.781401 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:49:48.781546 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.781555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.781618 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.781773 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.782802 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.784164 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.785955 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.786557 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.787147 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.790008 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.790013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.790014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.790026 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.792092 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.792631 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.792650 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.792650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.792769 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.793108 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.793109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.793110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.793420 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.793532 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.793847 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.793978 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1400,"thread_start_us":91,"threads_started":1}
I20260812 14:49:48.794165 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.779769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:34307)
0812 14:49:48.780018 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.780057 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.780661 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.781858 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.781862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.782359 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:48.782644 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.782651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.784311 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.784312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.785627 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.785629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.786680 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.786682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.786832 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.787486 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:48.787495 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:48.789890 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:48.792203 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.792211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.792213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.792545 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.792845 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.792846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.792846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.793015 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.793504 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:48.793513 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:48.793656 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.793774 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.793965 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
I20260812 14:49:48.794890 9011 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:48.795011 9011 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:48.795220 9011 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
I20260812 14:49:48.796046 8656 ts_manager.cc:194] Registered new tserver with Master: 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:48.797109 8656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:34307
I20260812 14:49:48.798211 9011 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:49:48.806960 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.796654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.0.0.1:39188)
0812 14:49:48.796862 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.796913 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.796989 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.797803 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.797809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.798132 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:48.798188 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.798191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.799634 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.799636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.801337 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.801340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.801945 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.801946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.801996 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.802503 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:48.802507 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:48.803969 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:48.805919 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.805925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.805927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.806122 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.806336 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.806337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.806337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.806396 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.806690 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:48.806695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:48.806739 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.806803 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.806892 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260812 14:49:48.807049 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.797438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39188 (local address 127.13.146.129:36819)
0812 14:49:48.797619 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.797624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.797636 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:49:48.797675 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.797677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.797697 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.797780 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.798300 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.799523 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.801415 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.801866 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.801940 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.804051 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.804060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.804060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.804067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.805842 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.806205 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.806208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.806208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.806272 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.806455 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.806456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.806456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.806625 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.806706 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.806857 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.806954 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260812 14:49:48.807654 8886 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a23a8b888d413580e2faa1549ad57b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:48 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:39188
I20260812 14:49:48.808350 9030 tablet_replica.cc:330] stopping tablet replica
I20260812 14:49:48.808462 9030 raft_consensus.cc:2229] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:49:48.808557 9030 raft_consensus.cc:2258] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:48.808881 9030 ts_tablet_manager.cc:1905] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:48.810927 9030 ts_tablet_manager.cc:1918] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:48.810995 9030 log.cc:1196] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/37a23a8b888d413580e2faa1549ad57b
I20260812 14:49:48.811237 9030 ts_tablet_manager.cc:1939] T 37a23a8b888d413580e2faa1549ad57b P 2dff3e56e8ff4a5b978aaf628f463c85: Deleting consensus metadata
I20260812 14:49:48.811717 8637 catalog_manager.cc:4841] TS 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819): tablet 37a23a8b888d413580e2faa1549ad57b (table test-table [id=f1e697a64de44ea0a3a9982c18578d19]) successfully deleted
W20260812 14:49:48.877811 9020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:48.878069 9020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:48.878109 9020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:48.879287 9020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:48.879360 9020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:48.879405 9020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:49:48.881173 9020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:36119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=44315
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:48.881484 9020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:48.881798 9020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:48.888432 9036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.888397 9037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:48.888453 9040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:48.889042 9020 server_base.cc:1034] running on GCE node
I20260812 14:49:48.889269 9020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:48.889478 9020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:48.890643 9020 hybrid_clock.cc:648] HybridClock initialized: now 1786546188890624 us; error 47 us; skew 500 ppm
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:48.893131 9020 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:49:48.893643 9020 webserver.cc:460] Webserver started at http://127.13.146.130:44315/ using document root <none> and password file <none>
I20260812 14:49:48.893890 9020 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:48.893947 9020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:48.895373 9020 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:48.896006 9052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.896204 9020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:48.896296 9020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "1d84c901b5ff4820a6ffe0c7064ce91b"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "d7201002e82ab1caf99af92a809f8094"
server_key_iv: "d7dd09f6bcb1a3e2bf4e4c957841cb86"
server_key_version: "encryptionkey@0"
I20260812 14:49:48.896626 9020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:48.907667 9020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:48.910012 9020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:48.910122 9020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:48.910291 9020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:48.910601 9020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:48.910636 9020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.910658 9020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:48.910714 9020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:48.916594 9020 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:36119
I20260812 14:49:48.916666 9247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:36119 every 8 connection(s)
I20260812 14:49:48.916982 9020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:49:48 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546188, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:48.925694 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 9020
I20260812 14:49:48.925953 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.917830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:34283 (local address 127.13.146.190:39727)
0812 14:49:48.917898 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:48.917900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:48.918113 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:49:48.918587 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:48.918588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:48.918598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:48.918650 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:48.919542 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.920025 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.920952 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.921140 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.921189 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.923235 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:48.923236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:48.923237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:48.923244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:48.924644 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.924992 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.924993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.924994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.925045 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:48.925309 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:48.925310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:48.925310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:48.925412 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:48.925460 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:48.925737 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 14:49:48.925837 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:49:48.925987 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:46671
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=33947
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:48.925987 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:48.917792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:34283)
0812 14:49:48.917983 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:48.918025 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:48.918504 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:48.918711 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:48.918714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:48.919261 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:48.919464 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.919469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.920107 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.920108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.920878 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:48.920880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:48.921257 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:48.921258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:48.921379 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:48.921808 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:48.921816 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:48.923149 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:48.924719 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.924725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.924726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.924926 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.925104 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:48.925105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:48.925105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:48.925200 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:48.925467 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:48.925472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:48.925592 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 14:49:48.925673 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:49:48.925822 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260812 14:49:48.926589 9249 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:48.926702 9249 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:48.926891 9249 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
I20260812 14:49:48.927327 8656 ts_manager.cc:194] Registered new tserver with Master: 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
I20260812 14:49:48.927860 8656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:34283
W20260812 14:49:49.009909 9257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:49.010197 9257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:49.010243 9257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:49.011500 9257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:49.011590 9257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:49.011634 9257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:49:49.013270 9257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:46671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=33947
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:49.013577 9257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:49.013827 9257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:49.019066 9269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.019237 9266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.019134 9274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.019353 9257 server_base.cc:1034] running on GCE node
I20260812 14:49:49.019567 9257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:49.019814 9257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:49.020978 9257 hybrid_clock.cc:648] HybridClock initialized: now 1786546189020974 us; error 38 us; skew 500 ppm
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:49.023679 9257 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:49:49.024194 9257 webserver.cc:460] Webserver started at http://127.13.146.131:33947/ using document root <none> and password file <none>
I20260812 14:49:49.024441 9257 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:49.024528 9257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:49.026011 9257 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:49:49.026666 9287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.026847 9257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:49:49.026958 9257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "196e826eda1d4d2eae3a0efb67e561c3"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "c081008f55c194149a26498792e3e1b4"
server_key_iv: "0c1af8bb5e3f0a234a4aaf26de082b1d"
server_key_version: "encryptionkey@0"
I20260812 14:49:49.027325 9257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:49.044727 9257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:49.046646 9257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:49.046811 9257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:49.047039 9257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:49.047366 9257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:49.047430 9257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.047465 9257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:49.047492 9257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.053932 9257 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:46671
I20260812 14:49:49.053982 9493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:46671 every 8 connection(s)
I20260812 14:49:49.054338 9257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:49:49.062659 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 9257
I20260812 14:49:49.064030 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.055059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:35231 (local address 127.13.146.190:39727)
0812 14:49:49.055145 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.055147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.055344 (+ 197us) server_negotiation.cc:407] Connection header received
0812 14:49:49.055829 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.055831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.055841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.055897 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:49.056872 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.057439 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.058406 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.058591 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.058643 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.060960 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:49.060961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:49.060961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:49.060969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:49.062378 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:49.062929 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:49.062930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:49.062931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:49.063001 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:49.063291 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:49.063291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:49.063292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:49.063453 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:49.063525 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.063887 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.063962 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 14:49:49.064414 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.055023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:35231)
0812 14:49:49.055212 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.055249 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.055737 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.055970 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:49.055973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:49.056572 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:49.056780 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.056785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.057538 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.057539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.058330 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.058333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.058667 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.058667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.058776 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.059338 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:49.059346 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:49.060862 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:49.062473 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:49.062482 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:49.062484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:49.062835 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:49.063075 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:49.063076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:49.063076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:49.063200 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:49.063531 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:49.063539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:49.063736 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 14:49:49.063866 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 14:49:49.064064 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260812 14:49:49.064918 9495 heartbeater.cc:347] Connected to a master server at 127.13.146.190:39727
I20260812 14:49:49.065047 9495 heartbeater.cc:464] Registering TS with master...
I20260812 14:49:49.065272 9495 heartbeater.cc:511] Master 127.13.146.190:39727 requested a full tablet report, sending...
I20260812 14:49:49.065698 8656 ts_manager.cc:194] Registered new tserver with Master: 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671)
I20260812 14:49:49.066159 8656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:35231
I20260812 14:49:49.069931 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.063162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.13.146.190:39727)
0812 14:49:49.063309 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.063311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.063389 (+ 78us) server_negotiation.cc:407] Connection header received
0812 14:49:49.063440 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.063440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.063453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.063513 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:49.064298 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.064931 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.065997 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.066289 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.066371 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.067625 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:49.067626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:49.067627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:49.067633 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:49.068916 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:49.069281 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:49.069282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:49.069283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:49.069347 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:49.069528 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:49.069528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:49.069529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:49.069638 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:49.069756 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.069794 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.069855 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":57,"threads_started":1}
I20260812 14:49:49.070652 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:49:49.073141 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.071014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.13.146.190:39727)
0812 14:49:49.071110 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.071111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.071158 (+ 47us) server_negotiation.cc:407] Connection header received
0812 14:49:49.071202 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.071202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.071230 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.071293 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:49:49.071378 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.071852 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.072533 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.072727 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.072781 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.072911 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:49:49.072997 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:49:49.073064 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.073080 (+ 16us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.073098 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 14:49:49.073458 8656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:49:49.075066 8656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:49:49.077440 8886 tablet_service.cc:1467] Processing CreateTablet for tablet bfb5450ecfeb4276843c2c62170703dd (DEFAULT_TABLE table=test-table [id=3bb4bb9f9cc34c5daf46b8a16d91da84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:49.077678 8886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb5450ecfeb4276843c2c62170703dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:49.079226 8816 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap starting.
I20260812 14:49:49.079562 8816 tablet_bootstrap.cc:654] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:49.080657 8816 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: No bootstrap required, opened a new log
I20260812 14:49:49.080729 8816 ts_tablet_manager.cc:1397] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:49:49.080876 8816 raft_consensus.cc:348] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.080971 8816 raft_consensus.cc:374] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:49.081022 8816 raft_consensus.cc:729] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dff3e56e8ff4a5b978aaf628f463c85, State: Initialized, Role: FOLLOWER
I20260812 14:49:49.081089 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.076955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:36119 (local address 127.0.0.1:51832)
0812 14:49:49.077071 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.077082 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.077117 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.077505 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:49.077506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:49.077508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:49.077553 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.077554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.078795 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.078795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.079783 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.079787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.080717 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.080718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.080797 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.080909 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 14:49:49.080970 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:49:49.081017 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":82}
I20260812 14:49:49.081079 8816 consensus_queue.cc:260] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.081193 8816 ts_tablet_manager.cc:1428] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.081486 9505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.077081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.13.146.130:36119)
0812 14:49:49.077336 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.077342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.077357 (+ 15us) server_negotiation.cc:407] Connection header received
0812 14:49:49.077390 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.077391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.077403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.077468 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:49.077653 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.078691 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.079893 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.080632 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.080702 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.080762 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.081362 (+ 600us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.081412 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":114,"threads_started":1}
I20260812 14:49:49.082201 9124 tablet_service.cc:1467] Processing CreateTablet for tablet bfb5450ecfeb4276843c2c62170703dd (DEFAULT_TABLE table=test-table [id=3bb4bb9f9cc34c5daf46b8a16d91da84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:49.082572 9124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb5450ecfeb4276843c2c62170703dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:49.082994 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.076955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:46671 (local address 127.0.0.1:36774)
0812 14:49:49.077142 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.077156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.077228 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.079448 (+ 2220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:49.079449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:49.079451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:49.079495 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.079496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.080690 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.080691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.081974 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.081976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.082825 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.082826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.082872 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.082875 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:49:49.082918 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:49:49.082947 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":21,"thread_start_us":92,"threads_started":1}
I20260812 14:49:49.083096 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.077143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.13.146.131:46671)
0812 14:49:49.079304 (+ 2161us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.079310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.079320 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:49:49.079358 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.079359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.079375 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.079451 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:49.079551 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.080545 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.082067 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.082663 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.082731 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.082783 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.082961 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.083008 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2118,"thread_start_us":115,"threads_started":1}
I20260812 14:49:49.083701 9376 tablet_service.cc:1467] Processing CreateTablet for tablet bfb5450ecfeb4276843c2c62170703dd (DEFAULT_TABLE table=test-table [id=3bb4bb9f9cc34c5daf46b8a16d91da84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:49:49.084024 9376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb5450ecfeb4276843c2c62170703dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:49:49.086758 9511 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Bootstrap starting.
I20260812 14:49:49.086866 9510 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Bootstrap starting.
I20260812 14:49:49.087361 9511 tablet_bootstrap.cc:654] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:49.087421 9510 tablet_bootstrap.cc:654] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Neither blocks nor log segments found. Creating new log.
I20260812 14:49:49.087862 9511 log.cc:824] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:49.087885 9510 log.cc:824] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:49.088599 9511 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: No bootstrap required, opened a new log
I20260812 14:49:49.088613 9510 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: No bootstrap required, opened a new log
I20260812 14:49:49.088677 9510 ts_tablet_manager.cc:1397] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:49:49.088685 9511 ts_tablet_manager.cc:1397] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:49:49.090201 9511 raft_consensus.cc:348] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.090224 9510 raft_consensus.cc:348] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.090355 9510 raft_consensus.cc:374] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:49.090358 9511 raft_consensus.cc:374] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:49:49.090376 9510 raft_consensus.cc:729] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d84c901b5ff4820a6ffe0c7064ce91b, State: Initialized, Role: FOLLOWER
I20260812 14:49:49.090408 9511 raft_consensus.cc:729] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196e826eda1d4d2eae3a0efb67e561c3, State: Initialized, Role: FOLLOWER
I20260812 14:49:49.090521 9510 consensus_queue.cc:260] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.090523 9511 consensus_queue.cc:260] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.090821 9511 ts_tablet_manager.cc:1428] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:49:49.090822 9510 ts_tablet_manager.cc:1428] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:49:49.090821 9495 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
I20260812 14:49:49.090822 9249 heartbeater.cc:503] Master 127.13.146.190:39727 was elected leader, sending a full tablet report...
W20260812 14:49:49.167683 9250 tablet.cc:2369] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:49:49.238296 9518 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:49.238441 9518 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.238922 9518 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:49.242015 9508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.239225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:44347 (local address 127.13.146.131:46671)
0812 14:49:49.239276 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.239278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.239310 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:49:49.239339 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.239340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.239349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.239435 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:49.239664 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.240271 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241256 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.241645 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241733 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.241743 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.241896 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.241931 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:49:49.242031 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.239122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:46671 (local address 127.13.146.130:44347)
0812 14:49:49.239187 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.239199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.239258 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.239464 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:49.239465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:49.239467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:49.239577 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.239580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.240365 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.240366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.241143 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.241145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.241733 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.241814 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.241817 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:49:49.241877 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:49:49.241919 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":17}
I20260812 14:49:49.242022 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.239245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:58077 (local address 127.13.146.129:36819)
0812 14:49:49.239290 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.239292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.239446 (+ 154us) server_negotiation.cc:407] Connection header received
0812 14:49:49.239503 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.239504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.239532 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.239611 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:49:49.239808 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.240308 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241286 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.241679 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241780 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.241857 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.241905 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.241950 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:49:49.242043 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.239123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.13.146.130:58077)
0812 14:49:49.239359 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.239367 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.239448 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.239628 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:49.239629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:49.239631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:49:49.239705 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.239706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.240394 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.240395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.241143 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:49.241145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.241779 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.241780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:49.241846 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.241849 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:49:49.241900 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:49:49.241943 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":29,"thread_start_us":127,"threads_started":1}
I20260812 14:49:49.242347 8926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb5450ecfeb4276843c2c62170703dd" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" is_pre_election: true
I20260812 14:49:49.242515 8926 raft_consensus.cc:2454] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 0.
I20260812 14:49:49.242563 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb5450ecfeb4276843c2c62170703dd" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196e826eda1d4d2eae3a0efb67e561c3" is_pre_election: true
I20260812 14:49:49.242712 9412 raft_consensus.cc:2454] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 0.
I20260812 14:49:49.242758 9058 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d84c901b5ff4820a6ffe0c7064ce91b, 2dff3e56e8ff4a5b978aaf628f463c85; no voters:
I20260812 14:49:49.242895 9518 raft_consensus.cc:2781] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:49:49.243000 9518 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:49:49.243028 9518 raft_consensus.cc:3037] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:49.243753 9518 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.243913 9518 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 election: Requested vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819)
I20260812 14:49:49.244069 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb5450ecfeb4276843c2c62170703dd" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196e826eda1d4d2eae3a0efb67e561c3"
I20260812 14:49:49.244086 8926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb5450ecfeb4276843c2c62170703dd" candidate_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
I20260812 14:49:49.244166 8926 raft_consensus.cc:3037] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:49.244171 9412 raft_consensus.cc:3037] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:49:49.244899 8926 raft_consensus.cc:2454] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 1.
I20260812 14:49:49.244904 9412 raft_consensus.cc:2454] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d84c901b5ff4820a6ffe0c7064ce91b in term 1.
I20260812 14:49:49.245074 9058 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d84c901b5ff4820a6ffe0c7064ce91b, 2dff3e56e8ff4a5b978aaf628f463c85; no voters:
I20260812 14:49:49.245187 9518 raft_consensus.cc:2781] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:49:49.245335 9518 raft_consensus.cc:686] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 LEADER]: Becoming Leader. State: Replica: 1d84c901b5ff4820a6ffe0c7064ce91b, State: Running, Role: LEADER
I20260812 14:49:49.245453 9518 consensus_queue.cc:237] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.246312 8656 catalog_manager.cc:5495] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130). New cstate: current_term: 1 leader_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } health_report { overall_health: UNKNOWN } } }
I20260812 14:49:49.262264 9505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.260061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.13.146.130:36119)
0812 14:49:49.260133 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:49.260135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:49.260146 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:49:49.260180 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:49.260181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:49.260199 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:49.260259 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:49:49.260379 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.260835 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.261548 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:49.261734 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:49.261790 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:49.261919 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:49:49.262011 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:49:49.262073 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:49.262107 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:49:49.262124 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 14:49:49.264164 8926 raft_consensus.cc:1261] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Refusing update from remote peer 1d84c901b5ff4820a6ffe0c7064ce91b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:49.264182 9412 raft_consensus.cc:1261] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Refusing update from remote peer 1d84c901b5ff4820a6ffe0c7064ce91b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:49:49.264521 9520 consensus_queue.cc:1035] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:49.264701 9518 consensus_queue.cc:1035] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [LEADER]: Connected to new peer: Peer: permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:49:49.267180 9522 mvcc.cc:205] Tried to move back new op lower bound from 7317693191223275520 to 7317693191150034944. Current Snapshot: MvccSnapshot[applied={T|T < 7317693191223275520}]
I20260812 14:49:49.267319 9523 mvcc.cc:205] Tried to move back new op lower bound from 7317693191223275520 to 7317693191150034944. Current Snapshot: MvccSnapshot[applied={T|T < 7317693191223275520}]
I20260812 14:49:49.270346 8655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:49.270444 8655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:49:49.271375 8655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Sending DeleteTablet for 3 replicas of tablet bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:49.271746 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 8722
I20260812 14:49:49.271715 9376 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb5450ecfeb4276843c2c62170703dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:36774
I20260812 14:49:49.271801 8886 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb5450ecfeb4276843c2c62170703dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:39188
I20260812 14:49:49.282101 9124 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb5450ecfeb4276843c2c62170703dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:49:49 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:51832
I20260812 14:49:49.291041 9545 tablet_replica.cc:330] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: stopping tablet replica
I20260812 14:49:49.291177 9545 raft_consensus.cc:2229] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:49:49.291303 9545 raft_consensus.cc:2258] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:49.291791 9545 ts_tablet_manager.cc:1905] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:49.293449 9545 ts_tablet_manager.cc:1918] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:49.293555 9545 log.cc:1196] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:49.293918 9545 ts_tablet_manager.cc:1939] T bfb5450ecfeb4276843c2c62170703dd P 1d84c901b5ff4820a6ffe0c7064ce91b: Deleting consensus metadata
I20260812 14:49:49.293989 9543 tablet_replica.cc:330] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: stopping tablet replica
I20260812 14:49:49.294137 9543 raft_consensus.cc:2229] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:49:49.294288 9543 raft_consensus.cc:2258] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:49:49.294531 8637 catalog_manager.cc:4841] TS 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): tablet bfb5450ecfeb4276843c2c62170703dd (table test-table [id=3bb4bb9f9cc34c5daf46b8a16d91da84]) successfully deleted
I20260812 14:49:49.294998 9543 ts_tablet_manager.cc:1905] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:49:49.296464 9543 ts_tablet_manager.cc:1918] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:49:49.296589 9543 log.cc:1196] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/bfb5450ecfeb4276843c2c62170703dd
W20260812 14:49:49.296727 8637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:36819 (error 108)
I20260812 14:49:49.296976 9543 ts_tablet_manager.cc:1939] T bfb5450ecfeb4276843c2c62170703dd P 196e826eda1d4d2eae3a0efb67e561c3: Deleting consensus metadata
I20260812 14:49:49.297529 8634 catalog_manager.cc:4841] TS 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): tablet bfb5450ecfeb4276843c2c62170703dd (table test-table [id=3bb4bb9f9cc34c5daf46b8a16d91da84]) successfully deleted
W20260812 14:49:49.297608 9504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:49:49.297305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36819 (local address 127.0.0.1:39196)
0812 14:49:49.297377 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.297394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.297460 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.297552 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36819: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 14:49:49.297724 8637 catalog_manager.cc:4584] TS 2dff3e56e8ff4a5b978aaf628f463c85 (127.13.146.129:36819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfb5450ecfeb4276843c2c62170703dd: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36819: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 14:49:49.298476 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 9020
I20260812 14:49:49.321346 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 9257
I20260812 14:49:49.337397 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 8604
I20260812 14:49:49.346189 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--webserver_interface=127.13.146.190
--webserver_port=35505
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:39727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:49:49.430514 9547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:49.430753 9547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:49.430778 9547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:49.431978 9547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:49:49.432024 9547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:49.432039 9547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:49.432050 9547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:49:49.432063 9547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:49:49.433682 9547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:39727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:39727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:49.433939 9547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:49.434194 9547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:49.439675 9563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.439675 9554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.439671 9558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.439844 9547 server_base.cc:1034] running on GCE node
I20260812 14:49:49.440341 9547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:49.440671 9547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:49.441824 9547 hybrid_clock.cc:648] HybridClock initialized: now 1786546189441800 us; error 38 us; skew 500 ppm
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:49.444572 9547 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:49:49.445087 9547 webserver.cc:460] Webserver started at http://127.13.146.190:35505/ using document root <none> and password file <none>
I20260812 14:49:49.445426 9547 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:49.445498 9547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:49.446902 9547 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:49.447618 9574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.447794 9547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:49:49.447902 9547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "be67d8a0701941ab96d0350161b7e029"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "611d870246d881962428dcc8f5548a2b"
server_key_iv: "50a577fee589a019b92aa793894236c8"
server_key_version: "encryptionkey@0"
I20260812 14:49:49.448293 9547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:49.478247 9547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:49.480798 9547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:49.480979 9547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:49.485388 9547 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:39727
I20260812 14:49:49.485430 9674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:39727 every 8 connection(s)
I20260812 14:49:49.485847 9547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:49:49.487457 9675 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:49:49.488185 9675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap starting.
I20260812 14:49:49.492363 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 9547
I20260812 14:49:49.492734 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:36819
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=33535
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:49.498333 9675 log.cc:824] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:49.500626 9675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:49:49.500950 9675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029: Bootstrap complete.
I20260812 14:49:49.503366 9675 raft_consensus.cc:348] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:49.503590 9675 raft_consensus.cc:729] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Initialized, Role: FOLLOWER
I20260812 14:49:49.503767 9675 consensus_queue.cc:260] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:49.503830 9675 raft_consensus.cc:388] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:49:49.503887 9675 raft_consensus.cc:482] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:49:49.503937 9675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:49:49.504944 9675 raft_consensus.cc:504] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:49.505067 9675 leader_election.cc:302] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be67d8a0701941ab96d0350161b7e029; no voters:
I20260812 14:49:49.505303 9675 leader_election.cc:288] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:49:49.505340 9681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:49:49.505456 9681 raft_consensus.cc:686] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [term 3 LEADER]: Becoming Leader. State: Replica: be67d8a0701941ab96d0350161b7e029, State: Running, Role: LEADER
I20260812 14:49:49.505632 9675 sys_catalog.cc:564] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:49:49.505628 9681 consensus_queue.cc:237] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } }
I20260812 14:49:49.506055 9685 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be67d8a0701941ab96d0350161b7e029. Latest consensus state: current_term: 3 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:49.506142 9685 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:49.506364 9683 sys_catalog.cc:455] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "be67d8a0701941ab96d0350161b7e029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be67d8a0701941ab96d0350161b7e029" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 39727 } } }
I20260812 14:49:49.506446 9683 sys_catalog.cc:458] T 00000000000000000000000000000000 P be67d8a0701941ab96d0350161b7e029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:49:49.506687 9691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:49:49.507622 9691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:49:49.507799 9691 catalog_manager.cc:1259] Loaded cluster ID: c40c364d567f4dc3ad041163a9ac7866
I20260812 14:49:49.507998 9691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 14:49:49.508291 9691 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7163afd9ad google::LogMessage::Fail() at ??:0
@ 0x7f7163b0088a google::LogMessage::SendToLog() at ??:0
@ 0x7f7163afd4de google::LogMessage::Flush() at ??:0
@ 0x7f7163afea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f716ba1e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f716ba278b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f71659cb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f71659c0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f71668246db start_thread at ??:0
@ 0x7f71625fb61f clone at ??:0
W20260812 14:49:49.593552 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:49.593775 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:49.593812 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:49.594964 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:49.595041 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:49.595088 9678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:49:49.596585 9678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:36819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=33535
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:49.596855 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:49.597092 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:49.602164 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.602170 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.602392 9678 server_base.cc:1034] running on GCE node
W20260812 14:49:49.602178 9739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.602710 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:49.602972 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:49.604149 9678 hybrid_clock.cc:648] HybridClock initialized: now 1786546189604132 us; error 32 us; skew 500 ppm
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:49.607259 9678 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:49:49.607790 9678 webserver.cc:460] Webserver started at http://127.13.146.129:33535/ using document root <none> and password file <none>
I20260812 14:49:49.608054 9678 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:49.608130 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:49.609557 9678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:49.610321 9758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.610527 9678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:49:49.610635 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "2dff3e56e8ff4a5b978aaf628f463c85"
format_stamp: "Formatted at 2026-08-12 14:49:47 on dist-test-slave-8hhm"
server_key: "b3fe2172a5522dbd1ab67f1fe53a05cc"
server_key_iv: "c8dde3adab959c46fbafc993c1f78f35"
server_key_version: "encryptionkey@0"
I20260812 14:49:49.611008 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:49.625280 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:49.627754 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:49.627923 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:49.628176 9678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:49.628695 9766 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:49:49.630818 9678 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:49:49.630936 9678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:49:49.630983 9678 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:49:49.631688 9678 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:49:49.631768 9678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:49:49.631856 9766 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap starting.
I20260812 14:49:49.639585 9678 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:36819
I20260812 14:49:49.639746 9951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:36819 every 8 connection(s)
I20260812 14:49:49.640038 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:49:49.643142 9766 log.cc:824] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Log is configured to *not* fsync() on all Append() calls
I20260812 14:49:49.646067 9766 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:49:49.646404 9766 tablet_bootstrap.cc:492] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Bootstrap complete.
I20260812 14:49:49.647751 9766 ts_tablet_manager.cc:1397] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260812 14:49:49.649344 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 9678
I20260812 14:49:49.649150 9766 raft_consensus.cc:348] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.649537 9766 raft_consensus.cc:729] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dff3e56e8ff4a5b978aaf628f463c85, State: Initialized, Role: FOLLOWER
I20260812 14:49:49.649675 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:36119
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=44315
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:49.649688 9766 consensus_queue.cc:260] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:49.649988 9766 ts_tablet_manager.cc:1428] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260812 14:49:49.709846 9955 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:49:49.641014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:36379)
0812 14:49:49.641364 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.641410 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:49.641893 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:49.709640 (+ 67747us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":266,"threads_started":1}
W20260812 14:49:49.710184 9953 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:39727 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:39727: Client connection negotiation failed: client connection to 127.13.146.190:39727: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:49:49.710556 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.710426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:43269)
0812 14:49:49.710481 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.710530 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:49:49.710937 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.710844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:55597)
0812 14:49:49.710891 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.710916 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:49:49.711090 9953 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:49:49.738080 9960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:49.738284 9960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:49.738308 9960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:49.739400 9960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:49.739437 9960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:49.739462 9960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:49:49.741325 9960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:36119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=44315
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:49.741516 9960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:49.741688 9960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:49.746853 9974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.746865 9965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.746868 9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.747231 9960 server_base.cc:1034] running on GCE node
I20260812 14:49:49.747432 9960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:49.747633 9960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:49.748814 9960 hybrid_clock.cc:648] HybridClock initialized: now 1786546189748816 us; error 42 us; skew 500 ppm
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:49.751665 9960 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:49:49.752162 9960 webserver.cc:460] Webserver started at http://127.13.146.130:44315/ using document root <none> and password file <none>
I20260812 14:49:49.752437 9960 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:49.752487 9960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:49.753866 9960 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:49.754510 9984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.754719 9960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:49.754794 9960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "1d84c901b5ff4820a6ffe0c7064ce91b"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "d7201002e82ab1caf99af92a809f8094"
server_key_iv: "d7dd09f6bcb1a3e2bf4e4c957841cb86"
server_key_version: "encryptionkey@0"
I20260812 14:49:49.755154 9960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:49.764544 9960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:49.766933 9960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:49.767048 9960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:49.767253 9960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:49.767573 9960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:49.767609 9960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.767632 9960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:49.767678 9960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.774290 9960 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:36119
I20260812 14:49:49.774390 10188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:36119 every 8 connection(s)
I20260812 14:49:49.774708 9960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:49:49.776036 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.775581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:49629)
0812 14:49:49.775774 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.775854 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260812 14:49:49.776340 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 9960
W20260812 14:49:49.776484 10192 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:39727 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:39727: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
I20260812 14:49:49.776649 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:46671
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:39727
--webserver_port=33947
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:49:49.776786 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.776682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:46683)
0812 14:49:49.776735 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.776762 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:49:49.777077 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.776988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:46071)
0812 14:49:49.777033 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.777056 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:49:49.777199 10192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:49:49.863101 10199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:49:49.863349 10199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:49:49.863389 10199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:49:49.864542 10199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:49:49.864609 10199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:49:49.864652 10199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:49:49.866227 10199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:46671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=33947
--tserver_master_addrs=127.13.146.190:39727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:49:49.866509 10199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:49:49.866757 10199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:49:49.872079 10206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:49:49.872169 10212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.872205 10199 server_base.cc:1034] running on GCE node
W20260812 14:49:49.872171 10215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:49:49.872495 10199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:49:49.872766 10199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:49:49.873939 10199 hybrid_clock.cc:648] HybridClock initialized: now 1786546189873930 us; error 36 us; skew 500 ppm
Aug 12 14:49:49 dist-test-slave-8hhm krb5kdc[7561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:49:49.876628 10199 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:49:49.877018 10199 webserver.cc:460] Webserver started at http://127.13.146.131:33947/ using document root <none> and password file <none>
I20260812 14:49:49.877296 10199 fs_manager.cc:359] Metadata directory not provided
I20260812 14:49:49.877347 10199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:49:49.878615 10199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:49:49.879175 10225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.879338 10199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:49:49.879433 10199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "196e826eda1d4d2eae3a0efb67e561c3"
format_stamp: "Formatted at 2026-08-12 14:49:48 on dist-test-slave-8hhm"
server_key: "c081008f55c194149a26498792e3e1b4"
server_key_iv: "0c1af8bb5e3f0a234a4aaf26de082b1d"
server_key_version: "encryptionkey@0"
I20260812 14:49:49.879765 10199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:49:49.906514 10199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:49:49.909024 10199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:49:49.909142 10199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:49:49.909404 10199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:49:49.909682 10199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:49:49.909722 10199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.909745 10199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:49:49.909758 10199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:49:49.915948 10199 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:46671
I20260812 14:49:49.916046 10426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:46671 every 8 connection(s)
I20260812 14:49:49.916390 10199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:49:49.917546 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.917139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:38097)
0812 14:49:49.917326 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.917404 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
W20260812 14:49:49.917961 10428 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:39727 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:39727: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
I20260812 14:49:49.918303 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.918206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:38141)
0812 14:49:49.918258 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.918286 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:49:49.918589 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:49.918538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:54541)
0812 14:49:49.918560 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:49.918577 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 14:49:49.918712 10428 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:49:49.922974 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 10199
I20260812 14:49:50.711755 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:50.711437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:39409)
0812 14:49:50.711624 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:50.711666 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":67,"threads_started":1}
I20260812 14:49:50.777750 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:50.777525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:47329)
0812 14:49:50.777645 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:50.777686 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260812 14:49:50.919293 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:50.919044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:40049)
0812 14:49:50.919205 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:50.919237 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260812 14:49:51.522087 10440 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:51.522223 10440 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:51.522513 10440 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
Aug 12 14:49:51 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.131@KRBTEST.COM
Aug 12 14:49:51 dist-test-slave-8hhm krb5kdc[7561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546189, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.130@KRBTEST.COM
I20260812 14:49:51.534052 10446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.522872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:45655 (local address 127.13.146.130:36119)
0812 14:49:51.523049 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:51.523061 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:51.523086 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:49:51.523508 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:51.523510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:51.523550 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:51.523666 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:51.524504 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.525522 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.526560 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.527208 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.527323 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:51.530358 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:51.530362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:51.530362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:51.530372 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:51.532265 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532640 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:51.532643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:51.532644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:51.532757 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532969 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:51.532970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:51.532971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:51.533320 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:51.533411 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:51.533794 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 14:49:51.533933 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260812 14:49:51.534054 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.522705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:46671 (local address 127.13.146.129:43785)
0812 14:49:51.522847 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:51.522858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:51.522920 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:51.523788 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:51.523790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:51.524181 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:51.524374 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:51.524380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.525579 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.525580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:51.526451 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:51.526454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.527342 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.527344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:51.527420 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:51.528093 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:51.528101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:51.529816 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:51.532120 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:51.532129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:51.532437 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:51.532738 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:51.532739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:51.532890 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:51.533400 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:51.533408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:51.533575 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 14:49:51.533723 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 14:49:51.533937 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":30,"thread_start_us":78,"threads_started":1}
I20260812 14:49:51.534070 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.522834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:43785 (local address 127.13.146.131:46671)
0812 14:49:51.523034 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:49:51.523046 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:49:51.523077 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:49:51.523518 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:49:51.523520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:49:51.523556 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:49:51.523713 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:49:51.524510 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.525463 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.526548 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.527220 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.527745 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:51.529933 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:49:51.529952 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:49:51.529952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:49:51.529964 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:49:51.532026 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532522 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:51.532525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:51.532525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:51.532658 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532970 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:49:51.532970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:49:51.532971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:49:51.533320 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:49:51.533411 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:49:51.533794 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 14:49:51.533933 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":62,"threads_started":1}
I20260812 14:49:51.534109 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.522707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:36119 (local address 127.13.146.129:45655)
0812 14:49:51.522866 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:51.522879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:49:51.522968 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:49:51.523786 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:49:51.523789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:49:51.524180 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:49:51.524374 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:51.524380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.525606 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.525607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:51.526451 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:49:51.526454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:49:51.527297 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:49:51.527298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:49:51.527415 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:49:51.528167 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:49:51.528171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:49:51.530247 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:49:51.532345 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:51.532347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:51.532563 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:51.532831 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:49:51.532831 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:49:51.532832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:49:51.532898 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:49:51.533400 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:49:51.533408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:49:51.533575 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 14:49:51.533714 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 14:49:51.533997 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":183,"thread_start_us":83,"threads_started":1}
W20260812 14:49:51.535202 9760 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
W20260812 14:49:51.535606 9762 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:51.535655 9762 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dff3e56e8ff4a5b978aaf628f463c85; no voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b
I20260812 14:49:51.535833 10440 raft_consensus.cc:2726] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:49:51.712538 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.712349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:47165)
0812 14:49:51.712444 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:51.712489 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 14:49:51.778517 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.778237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:59477)
0812 14:49:51.778397 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:51.778458 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":63,"threads_started":1}
I20260812 14:49:51.919965 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:51.919727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:44067)
0812 14:49:51.919870 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:51.919917 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260812 14:49:52.713333 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:52.713059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:44823)
0812 14:49:52.713234 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:52.713281 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260812 14:49:52.779189 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:52.778957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:40581)
0812 14:49:52.779104 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:52.779147 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260812 14:49:52.920836 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:52.920615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:46275)
0812 14:49:52.920746 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:52.920788 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260812 14:49:53.521783 10457 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:53.521888 10457 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:53.522102 10457 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
W20260812 14:49:53.522606 9762 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
W20260812 14:49:53.522696 9760 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:53.522799 9760 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dff3e56e8ff4a5b978aaf628f463c85; no voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b
I20260812 14:49:53.522891 10457 raft_consensus.cc:2726] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:49:53.714172 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:53.713878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:46845)
0812 14:49:53.714053 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:53.714114 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260812 14:49:53.779942 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:53.779640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:41493)
0812 14:49:53.779839 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:53.779883 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260812 14:49:53.921584 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:53.921355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:44687)
0812 14:49:53.921478 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:53.921516 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260812 14:49:54.715076 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:54.714820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:41987)
0812 14:49:54.714989 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:54.715027 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260812 14:49:54.780694 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:54.780453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:36325)
0812 14:49:54.780583 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:54.780645 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260812 14:49:54.922272 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:54.922058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:38703)
0812 14:49:54.922181 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:54.922225 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260812 14:49:55.361367 10468 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:55.361470 10468 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:55.361718 10468 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
W20260812 14:49:55.362146 9762 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
W20260812 14:49:55.362226 9760 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:55.362303 9760 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dff3e56e8ff4a5b978aaf628f463c85; no voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b
I20260812 14:49:55.362396 10468 raft_consensus.cc:2726] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:49:55.715823 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:55.715561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:52039)
0812 14:49:55.715740 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:55.715772 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260812 14:49:55.781435 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:55.781158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:44853)
0812 14:49:55.781343 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:55.781385 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260812 14:49:55.923041 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:55.922818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:33963)
0812 14:49:55.922953 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:55.922994 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260812 14:49:56.716501 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:56.716276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:35547)
0812 14:49:56.716406 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:56.716449 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260812 14:49:56.782223 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:56.781959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:36531)
0812 14:49:56.782118 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:56.782177 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260812 14:49:56.923849 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:56.923633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:53251)
0812 14:49:56.923764 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:56.923802 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260812 14:49:57.717255 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:57.716984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:52965)
0812 14:49:57.717110 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:57.717161 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260812 14:49:57.782912 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:57.782677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:35225)
0812 14:49:57.782816 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:57.782864 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260812 14:49:57.924681 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:57.924445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:41131)
0812 14:49:57.924589 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:57.924632 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260812 14:49:58.327136 10484 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:49:58.327258 10484 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:49:58.327461 10484 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
W20260812 14:49:58.327881 9760 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
W20260812 14:49:58.327936 9762 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
I20260812 14:49:58.327963 9760 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dff3e56e8ff4a5b978aaf628f463c85; no voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b
I20260812 14:49:58.328037 10484 raft_consensus.cc:2726] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:49:58.717944 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:58.717717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:59517)
0812 14:49:58.717859 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:58.717892 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260812 14:49:58.783682 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:58.783422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:40759)
0812 14:49:58.783568 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:58.783617 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260812 14:49:58.925586 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:58.925347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:40327)
0812 14:49:58.925496 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:58.925538 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260812 14:49:59.718686 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:59.718431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:56093)
0812 14:49:59.718589 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:59.718635 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260812 14:49:59.784385 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:59.784148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:40779)
0812 14:49:59.784289 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:59.784336 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260812 14:49:59.926349 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:49:59.926087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:51919)
0812 14:49:59.926253 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:49:59.926294 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260812 14:50:00.719377 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:00.719156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:44511)
0812 14:50:00.719290 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:00.719328 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260812 14:50:00.785169 10493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:00.784910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:58789)
0812 14:50:00.785077 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:00.785119 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260812 14:50:00.926998 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:00.926793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:60199)
0812 14:50:00.926918 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:00.926950 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260812 14:50:01.720165 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:01.719856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:60861)
0812 14:50:01.720047 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:01.720082 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
I20260812 14:50:01.786026 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:01.785744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:50957)
0812 14:50:01.785887 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:01.785934 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260812 14:50:01.927737 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:01.927479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:45735)
0812 14:50:01.927629 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:01.927670 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260812 14:50:02.721040 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:02.720776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:58425)
0812 14:50:02.720939 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:02.720988 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260812 14:50:02.786830 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:02.786561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:48583)
0812 14:50:02.786743 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:02.786777 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260812 14:50:02.928582 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:02.928330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:52305)
0812 14:50:02.928488 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:02.928532 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260812 14:50:03.721771 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:03.721543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:57449)
0812 14:50:03.721681 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:03.721721 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260812 14:50:03.758859 10507 raft_consensus.cc:482] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:03.758949 10507 raft_consensus.cc:504] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "196e826eda1d4d2eae3a0efb67e561c3" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 46671 } } peers { permanent_uuid: "1d84c901b5ff4820a6ffe0c7064ce91b" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36119 } } peers { permanent_uuid: "2dff3e56e8ff4a5b978aaf628f463c85" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36819 } }
I20260812 14:50:03.759122 10507 leader_election.cc:288] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671), 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119)
W20260812 14:50:03.759527 9762 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d84c901b5ff4820a6ffe0c7064ce91b (127.13.146.130:36119): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
W20260812 14:50:03.759613 9760 leader_election.cc:341] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 196e826eda1d4d2eae3a0efb67e561c3 (127.13.146.131:46671): Not found: Tablet not found: bfb5450ecfeb4276843c2c62170703dd
I20260812 14:50:03.759703 9760 leader_election.cc:302] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dff3e56e8ff4a5b978aaf628f463c85; no voters: 196e826eda1d4d2eae3a0efb67e561c3, 1d84c901b5ff4820a6ffe0c7064ce91b
I20260812 14:50:03.759781 10507 raft_consensus.cc:2726] T bfb5450ecfeb4276843c2c62170703dd P 2dff3e56e8ff4a5b978aaf628f463c85 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:50:03.787650 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:03.787393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:40071)
0812 14:50:03.787549 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:03.787598 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260812 14:50:03.929414 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:03.929103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:46001)
0812 14:50:03.929313 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:03.929360 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
I20260812 14:50:04.722596 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:04.722303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.129:38707)
0812 14:50:04.722492 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:04.722548 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260812 14:50:04.788456 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:04.788212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.130:38491)
0812 14:50:04.788368 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:04.788407 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260812 14:50:04.925619 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 9678
I20260812 14:50:04.932216 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:04.931894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:39727 (local address 127.13.146.131:47571)
0812 14:50:04.932111 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:04.932150 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":64,"threads_started":1}
I20260812 14:50:04.937970 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 9960
I20260812 14:50:04.953329 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 10199
2026-08-12T14:50:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18855 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:50:05 dist-test-slave-8hhm krb5kdc[10529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:50:05 dist-test-slave-8hhm krb5kdc[10529](info): set up 2 sockets
Aug 12 14:50:05 dist-test-slave-8hhm krb5kdc[10529](info): commencing operation
krb5kdc: starting...
W20260812 14:50:06.295027 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:50:06.427860 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:41411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:06.518431 10551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:06.518662 10551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:06.518736 10551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:06.519959 10551 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:50:06.520017 10551 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:50:06.520031 10551 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:50:06.520043 10551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:50:06.520056 10551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:06.520069 10551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:06.520082 10551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:50:06.520135 10551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:50:06.521763 10551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:41411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:06.522064 10551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:06.522321 10551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:06.527791 10562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:06.528002 10551 server_base.cc:1034] running on GCE node
W20260812 14:50:06.527810 10565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:06.527791 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:06.528457 10551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:06.528817 10551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:06.529978 10551 hybrid_clock.cc:648] HybridClock initialized: now 1786546206529951 us; error 41 us; skew 500 ppm
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:06.536438 10551 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:50:06.537009 10551 webserver.cc:460] Webserver started at http://127.13.146.190:38097/ using document root <none> and password file <none>
I20260812 14:50:06.537364 10551 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:06.537451 10551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:06.537640 10551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:50:06.538913 10551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "22ef993286194dda890a7cb7bfb3f61e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "f741967bf1c3bcf6e1ffb49f3444ba13"
server_key_iv: "cb0012025f425bdf696fe23fd867da8b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.539387 10551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "22ef993286194dda890a7cb7bfb3f61e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "f741967bf1c3bcf6e1ffb49f3444ba13"
server_key_iv: "cb0012025f425bdf696fe23fd867da8b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.540993 10551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:50:06.541977 10573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.542256 10551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:06.542340 10551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "22ef993286194dda890a7cb7bfb3f61e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "f741967bf1c3bcf6e1ffb49f3444ba13"
server_key_iv: "cb0012025f425bdf696fe23fd867da8b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.542418 10551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:06.553557 10551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:06.555790 10551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:06.555955 10551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:06.560544 10551 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:41411
I20260812 14:50:06.560576 10664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:41411 every 8 connection(s)
I20260812 14:50:06.560959 10551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:50:06.562016 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:06.564673 10667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap starting.
I20260812 14:50:06.565420 10667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:06.565766 10667 log.cc:824] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:06.566583 10667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: No bootstrap required, opened a new log
I20260812 14:50:06.568418 10667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:06.568593 10667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:06.568619 10667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Initialized, Role: FOLLOWER
I20260812 14:50:06.568812 10667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:06.568885 10667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:50:06.568933 10667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:50:06.568990 10667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:06.569705 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 10551
I20260812 14:50:06.569705 10667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:06.569801 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:50:06.569840 10667 leader_election.cc:302] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22ef993286194dda890a7cb7bfb3f61e; no voters:
I20260812 14:50:06.570037 13898 external_mini_cluster.cc:1428] Setting key dd6bbc51dbe996dccbd59eb51e6e9039
I20260812 14:50:06.570065 10667 leader_election.cc:288] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:50:06.570128 10674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:50:06.570341 10674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 LEADER]: Becoming Leader. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Running, Role: LEADER
I20260812 14:50:06.570367 10667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:50:06.570525 10674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:06.571523 10675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:06.571619 10675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
I20260812 14:50:06.572173 10677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22ef993286194dda890a7cb7bfb3f61e. Latest consensus state: current_term: 1 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:06.572260 10677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
W20260812 14:50:06.572860 10708 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:50:06.572924 10708 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:50:06.572985 10699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:50:06.573206 10699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:50:06.574995 10699 catalog_manager.cc:1347] Generated new cluster ID: 705fd7227ab3455692afdf4570ea2bac
I20260812 14:50:06.575071 10699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:06.582616 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:06.570779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.13.146.190:41411)
0812 14:50:06.570996 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:06.571006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:06.571049 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:50:06.571173 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:06.571181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:06.571242 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:06.571390 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:06.572421 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.573664 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.574647 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.575235 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.575351 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:06.578454 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:06.578459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:06.578459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:06.578475 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:06.580716 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.581165 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.581168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.581169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.581345 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.581581 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.581582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.581583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.581913 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:06.582062 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:06.582202 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:50:06.582354 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260812 14:50:06.588136 10699 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:50:06.588867 10699 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:50:06.593151 10699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Generated new TSK 0
I20260812 14:50:06.593358 10699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:50:06.628124 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:06.714116 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:06.714339 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:06.714365 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:06.715553 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:06.715598 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:06.715653 10715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:50:06.717096 10715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:06.717411 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:06.717669 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:06.722846 10727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:06.722875 10724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:06.723042 10715 server_base.cc:1034] running on GCE node
W20260812 14:50:06.723091 10723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:06.723356 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:06.723654 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:06.724820 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786546206724798 us; error 50 us; skew 500 ppm
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:06.727581 10715 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:50:06.727967 10715 webserver.cc:460] Webserver started at http://127.13.146.129:36689/ using document root <none> and password file <none>
I20260812 14:50:06.728160 10715 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:06.728204 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:06.728296 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:50:06.729384 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "c01c9d2f1a26d355e187640cd422cced"
server_key_iv: "33edff7061fe47a3f677a5ab3ada6a0b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.729708 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "c01c9d2f1a26d355e187640cd422cced"
server_key_iv: "33edff7061fe47a3f677a5ab3ada6a0b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.731010 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:06.731724 10738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.731930 10715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:06.732018 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "c01c9d2f1a26d355e187640cd422cced"
server_key_iv: "33edff7061fe47a3f677a5ab3ada6a0b"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.732075 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:06.743009 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:06.745409 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:06.745568 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:06.745820 10715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:06.746093 10715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:06.746125 10715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.746150 10715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:06.746162 10715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.752285 10715 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:43673
I20260812 14:50:06.752370 10959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:43673 every 8 connection(s)
I20260812 14:50:06.752700 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:50:06.754187 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 10715
I20260812 14:50:06.754274 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:50:06.754472 13898 external_mini_cluster.cc:1428] Setting key ea36b705300cf97fcbad4e26fe08e6c7
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
I20260812 14:50:06.763971 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:06.753584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:55789 (local address 127.13.146.190:41411)
0812 14:50:06.753646 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:06.753648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:06.753902 (+ 254us) server_negotiation.cc:407] Connection header received
0812 14:50:06.754538 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:06.754540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:06.754559 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:06.754650 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:06.755850 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.756596 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.758268 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.758469 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.758556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:06.761006 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:06.761008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:06.761008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:06.761019 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:06.762517 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.762965 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.762967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.762967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.763031 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.763299 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.763300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.763300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.763418 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:06.763492 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:06.763788 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:50:06.763881 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:50:06.764204 10962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:06.753557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:55789)
0812 14:50:06.753756 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:06.753804 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:06.754429 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:06.754766 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:06.754771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:06.755539 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:06.755748 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:06.755753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.756725 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.756727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:06.758178 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:06.758182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.758565 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.758566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:06.758702 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:06.759267 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:06.759276 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:06.760904 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:06.762623 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:06.762631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:06.762634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:06.762871 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:06.763104 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:06.763105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:06.763105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:06.763228 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:06.763497 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:06.763505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:06.763651 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:50:06.763751 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:50:06.763914 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":114,"threads_started":1}
I20260812 14:50:06.764686 10960 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:06.764801 10960 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:06.764986 10960 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:06.765601 10602 ts_manager.cc:194] Registered new tserver with Master: 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:06.766369 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:55789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:50:06.784271 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:06.871083 10967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:06.871299 10967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:06.871325 10967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:06.872562 10967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:06.872608 10967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:06.872635 10967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:50:06.874185 10967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:06.874383 10967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:06.874567 10967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:06.879892 10976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:06.879971 10978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:06.879892 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:06.880075 10967 server_base.cc:1034] running on GCE node
I20260812 14:50:06.880391 10967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:06.880692 10967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:06.881822 10967 hybrid_clock.cc:648] HybridClock initialized: now 1786546206881785 us; error 46 us; skew 500 ppm
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:06.885442 10967 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:50:06.886037 10967 webserver.cc:460] Webserver started at http://127.13.146.130:45109/ using document root <none> and password file <none>
I20260812 14:50:06.886394 10967 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:06.886474 10967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:06.886624 10967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:50:06.888170 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "2492c2247bda4c999231969d2a2d13d3"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "e011ce707b91f1a4c28c7d961802a5d7"
server_key_iv: "838e7d0ab3beadf725fcede5f148766e"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.888628 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "2492c2247bda4c999231969d2a2d13d3"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "e011ce707b91f1a4c28c7d961802a5d7"
server_key_iv: "838e7d0ab3beadf725fcede5f148766e"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.890180 10967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 14:50:06.891009 10993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.891208 10967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:06.891314 10967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "2492c2247bda4c999231969d2a2d13d3"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "e011ce707b91f1a4c28c7d961802a5d7"
server_key_iv: "838e7d0ab3beadf725fcede5f148766e"
server_key_version: "encryptionkey@0"
I20260812 14:50:06.891402 10967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:06.916096 10967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:06.918927 10967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:06.919108 10967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:06.919427 10967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:06.919834 10967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:06.919886 10967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.919924 10967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:06.919953 10967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:06.927942 10967 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:42689
I20260812 14:50:06.927983 11208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:42689 every 8 connection(s)
I20260812 14:50:06.928412 10967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:50:06.930797 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 10967
I20260812 14:50:06.930922 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:50:06.931137 13898 external_mini_cluster.cc:1428] Setting key ca3be45a51bbdb8ee8a657bc32288ffd
Aug 12 14:50:06 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546206, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:06.941428 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:06.929284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:51663 (local address 127.13.146.190:41411)
0812 14:50:06.929345 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:06.929347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:06.929601 (+ 254us) server_negotiation.cc:407] Connection header received
0812 14:50:06.930165 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:06.930167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:06.930180 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:06.930231 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:06.931601 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.932467 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.934098 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.934436 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.934512 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:06.937557 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:06.937560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:06.937560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:06.937569 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:06.939559 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.940131 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.940133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.940133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.940205 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:06.940537 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:06.940538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:06.940538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:06.940701 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:06.940771 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:06.941169 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 14:50:06.941294 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
I20260812 14:50:06.942111 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:06.929261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:51663)
0812 14:50:06.929447 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:06.929500 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:06.930046 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:06.930309 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:06.930313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:06.931235 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:06.931481 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:06.931507 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.932586 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.932588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:06.933973 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:06.933977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:06.934536 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:06.934537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:06.934704 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:06.935338 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:06.935349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:06.937433 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:06.939660 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:06.939669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:06.939671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:06.940011 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:06.940285 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:06.940286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:06.940286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:06.940446 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:06.940772 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:06.940779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:06.940962 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 14:50:06.941095 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 14:50:06.941313 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260812 14:50:06.942682 11210 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:06.942793 11210 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:06.942994 11210 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:06.943418 10602 ts_manager.cc:194] Registered new tserver with Master: 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
I20260812 14:50:06.943814 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:51663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:50:06.967741 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:07.056573 11217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:07.056834 11217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:07.056860 11217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:07.058178 11217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:07.058229 11217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:07.058257 11217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:50:07.059809 11217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:07.060101 11217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:07.060350 11217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:07.065744 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:07.065737 11217 server_base.cc:1034] running on GCE node
W20260812 14:50:07.065718 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:07.065713 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:07.066201 11217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:07.066470 11217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:07.067625 11217 hybrid_clock.cc:648] HybridClock initialized: now 1786546207067595 us; error 52 us; skew 500 ppm
Aug 12 14:50:07 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546207, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:07.070533 11217 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:50:07.071062 11217 webserver.cc:460] Webserver started at http://127.13.146.131:39361/ using document root <none> and password file <none>
I20260812 14:50:07.071386 11217 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:07.071460 11217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:07.071609 11217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:50:07.072868 11217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "23d2851107984eeda2a52e24feec9e3c"
format_stamp: "Formatted at 2026-08-12 14:50:07 on dist-test-slave-8hhm"
server_key: "f11a29c31ba0678c8038a16f48ff76f6"
server_key_iv: "65e0222473c77e3481f9514ec00d31ba"
server_key_version: "encryptionkey@0"
I20260812 14:50:07.073336 11217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "23d2851107984eeda2a52e24feec9e3c"
format_stamp: "Formatted at 2026-08-12 14:50:07 on dist-test-slave-8hhm"
server_key: "f11a29c31ba0678c8038a16f48ff76f6"
server_key_iv: "65e0222473c77e3481f9514ec00d31ba"
server_key_version: "encryptionkey@0"
I20260812 14:50:07.074851 11217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:07.075745 11243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:07.075964 11217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:07.076083 11217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "23d2851107984eeda2a52e24feec9e3c"
format_stamp: "Formatted at 2026-08-12 14:50:07 on dist-test-slave-8hhm"
server_key: "f11a29c31ba0678c8038a16f48ff76f6"
server_key_iv: "65e0222473c77e3481f9514ec00d31ba"
server_key_version: "encryptionkey@0"
I20260812 14:50:07.076175 11217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:07.097457 11217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:07.099448 11217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:07.099567 11217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:07.099776 11217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:07.100039 11217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:07.100069 11217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:07.100091 11217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:07.100104 11217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:07.106243 11217 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41815
I20260812 14:50:07.106294 11458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41815 every 8 connection(s)
I20260812 14:50:07.106642 11217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:50:07 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546207, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:07.114486 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 11217
I20260812 14:50:07.114601 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:50:07.114799 13898 external_mini_cluster.cc:1428] Setting key db3003e9318a4da6aa128b4562d55cdc
I20260812 14:50:07.117511 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.107450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:59419 (local address 127.13.146.190:41411)
0812 14:50:07.107517 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.107519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.107711 (+ 192us) server_negotiation.cc:407] Connection header received
0812 14:50:07.108193 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.108194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.108204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.108272 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:07.109264 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.109747 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.111254 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.111441 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.111499 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.114126 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:07.114127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:07.114128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:07.114135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:07.116046 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:07.116432 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:07.116433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:07.116433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:07.116490 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:07.116764 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:07.116766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:07.116766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:07.116920 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:07.117000 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.117304 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.117391 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:50:07.117592 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.107416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:59419)
0812 14:50:07.107570 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.107614 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.108105 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.108329 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.108332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.108929 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:07.109128 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.109134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.109830 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.109831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.111167 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.111170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.111519 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.111520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.111618 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.112121 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:07.112131 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:07.114005 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:07.116121 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:07.116128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:07.116131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:07.116364 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:07.116551 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:07.116552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:07.116552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:07.116670 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:07.116994 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:07.116998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:07.117116 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.117201 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.117396 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260812 14:50:07.118139 11459 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:07.118258 11459 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:07.118498 11459 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:07.118973 10602 ts_manager.cc:194] Registered new tserver with Master: 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815)
I20260812 14:50:07.119522 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:59419
I20260812 14:50:07.126871 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:50:07.135185 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.128288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.13.146.190:41411)
0812 14:50:07.128385 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.128388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.128490 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:50:07.128598 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.128599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.128611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.128703 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:07.129429 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.129984 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.130817 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.131045 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.131129 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.132424 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:07.132426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:07.132427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:07.132435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:07.133810 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:07.134206 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:07.134207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:07.134207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:07.134277 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:07.134535 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:07.134536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:07.134537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:07.134709 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:07.134825 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.134980 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.135073 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 14:50:07.137323 10602 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:50:07.139034 10602 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:50:07.146426 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.142164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.13.146.130:42689)
0812 14:50:07.142354 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.142359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.142372 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:50:07.142417 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.142419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.142438 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.142734 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:07.142809 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143522 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.145195 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.145951 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.146023 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.146083 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.146280 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.146327 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260812 14:50:07.146426 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.141657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42689 (local address 127.0.0.1:57040)
0812 14:50:07.141826 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.141868 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.141939 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.142585 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.142586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.142589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:07.142646 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.142648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143672 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.143673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.145096 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.145100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.146057 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.146058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.146125 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.146138 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.146254 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.146339 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":77,"thread_start_us":116,"threads_started":1}
I20260812 14:50:07.146428 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.141652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:41815 (local address 127.0.0.1:49560)
0812 14:50:07.141826 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.141868 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.141939 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.142724 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.142725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.142727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:07.142762 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.142763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143969 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.143970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.144990 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.144992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.146117 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.146118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.146195 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.146199 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.146253 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.146328 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":38,"thread_start_us":66,"threads_started":1}
I20260812 14:50:07.146996 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.141954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.13.146.131:41815)
0812 14:50:07.142485 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.142488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.142500 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:50:07.142540 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.142542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.142558 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.142648 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:07.142853 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143871 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.145074 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.145875 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.146016 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.146083 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.146467 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.146510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":108,"threads_started":1}
I20260812 14:50:07.147578 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.141660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.0.0.1:45220)
0812 14:50:07.142041 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.142049 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.142088 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.142328 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.142332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.142335 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:07.142409 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.142412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143739 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.143739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.145096 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.145100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.147307 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.147308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.147385 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.147389 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.147471 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.147513 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":78,"thread_start_us":112,"threads_started":1}
I20260812 14:50:07.147460 11090 tablet_service.cc:1467] Processing CreateTablet for tablet f68d1d9b09d94fe29db2560452f7b482 (DEFAULT_TABLE table=test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:07.147617 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.141954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.13.146.129:43673)
0812 14:50:07.142153 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.142158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.142167 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:50:07.142212 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.142214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.142226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.142317 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:07.142501 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.143646 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.146124 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.147144 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.147215 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.147271 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.147492 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.147535 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260812 14:50:07.147646 11328 tablet_service.cc:1467] Processing CreateTablet for tablet f68d1d9b09d94fe29db2560452f7b482 (DEFAULT_TABLE table=test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:07.147876 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68d1d9b09d94fe29db2560452f7b482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:07.147926 11328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68d1d9b09d94fe29db2560452f7b482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:07.148191 10834 tablet_service.cc:1467] Processing CreateTablet for tablet f68d1d9b09d94fe29db2560452f7b482 (DEFAULT_TABLE table=test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:07.148478 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68d1d9b09d94fe29db2560452f7b482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:07.151430 11496 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap starting.
I20260812 14:50:07.151619 11499 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Bootstrap starting.
I20260812 14:50:07.152271 11499 tablet_bootstrap.cc:654] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:07.152271 11496 tablet_bootstrap.cc:654] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:07.152912 11496 log.cc:824] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:07.152912 11499 log.cc:824] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:07.153344 11497 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Bootstrap starting.
I20260812 14:50:07.153852 11499 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: No bootstrap required, opened a new log
I20260812 14:50:07.153869 11496 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: No bootstrap required, opened a new log
I20260812 14:50:07.153932 11496 ts_tablet_manager.cc:1397] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:50:07.153934 11499 ts_tablet_manager.cc:1397] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:50:07.154006 11497 tablet_bootstrap.cc:654] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:07.154470 11497 log.cc:824] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:07.155164 11497 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: No bootstrap required, opened a new log
I20260812 14:50:07.155244 11497 ts_tablet_manager.cc:1397] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:50:07.155951 11499 raft_consensus.cc:348] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.155951 11496 raft_consensus.cc:348] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.156147 11499 raft_consensus.cc:374] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:07.156147 11496 raft_consensus.cc:374] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:07.156188 11496 raft_consensus.cc:729] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e6ea22be8b490d91bcb2153d06cd7e, State: Initialized, Role: FOLLOWER
I20260812 14:50:07.156188 11499 raft_consensus.cc:729] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2492c2247bda4c999231969d2a2d13d3, State: Initialized, Role: FOLLOWER
I20260812 14:50:07.156309 11496 consensus_queue.cc:260] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.156309 11499 consensus_queue.cc:260] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.156601 11499 ts_tablet_manager.cc:1428] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:50:07.156601 11496 ts_tablet_manager.cc:1428] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:50:07.156741 11210 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
I20260812 14:50:07.156836 10960 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
I20260812 14:50:07.156857 11497 raft_consensus.cc:348] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.157014 11497 raft_consensus.cc:374] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:07.157049 11497 raft_consensus.cc:729] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d2851107984eeda2a52e24feec9e3c, State: Initialized, Role: FOLLOWER
I20260812 14:50:07.157186 11497 consensus_queue.cc:260] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.157529 11497 ts_tablet_manager.cc:1428] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:50:07.157577 11459 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
W20260812 14:50:07.179001 11212 tablet.cc:2369] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:50:07.253500 10961 tablet.cc:2369] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:50:07.284924 11506 raft_consensus.cc:482] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:07.285113 11506 raft_consensus.cc:504] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.285624 11506 leader_election.cc:288] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689), 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:07.289199 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.285948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:54671 (local address 127.13.146.129:43673)
0812 14:50:07.286001 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.286003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.286146 (+ 143us) server_negotiation.cc:407] Connection header received
0812 14:50:07.286186 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.286188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.286198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.286266 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:07.286426 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.287246 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288454 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.288837 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288928 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.288938 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.289088 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.289127 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:50:07.289211 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.285869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.13.146.131:54671)
0812 14:50:07.286070 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.286079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.286133 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.286291 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.286291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.286292 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:07.286350 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.286352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.287355 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.287356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.288346 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.288349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.288928 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.289007 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.289011 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.289079 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.289122 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":136,"thread_start_us":113,"threads_started":1}
I20260812 14:50:07.289328 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.285869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42689 (local address 127.13.146.131:48695)
0812 14:50:07.285910 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.285923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:07.285972 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:07.286209 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:07.286209 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:07.286210 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:07.286300 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.286302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.287377 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.287378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.288338 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:07.288341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.288970 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:07.289074 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.289077 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:50:07.289148 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:50:07.289191 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":2}
I20260812 14:50:07.289561 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.285948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:48695 (local address 127.13.146.130:42689)
0812 14:50:07.286001 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.286003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.286016 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:50:07.286052 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.286053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.286071 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.286166 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:07.286401 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.287253 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288446 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.288878 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.288973 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.288982 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.289168 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.289228 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:50:07.289710 10870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d1d9b09d94fe29db2560452f7b482" candidate_uuid: "23d2851107984eeda2a52e24feec9e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" is_pre_election: true
I20260812 14:50:07.289932 10870 raft_consensus.cc:2454] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d2851107984eeda2a52e24feec9e3c in term 0.
I20260812 14:50:07.290051 11125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d1d9b09d94fe29db2560452f7b482" candidate_uuid: "23d2851107984eeda2a52e24feec9e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2492c2247bda4c999231969d2a2d13d3" is_pre_election: true
I20260812 14:50:07.290230 11125 raft_consensus.cc:2454] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d2851107984eeda2a52e24feec9e3c in term 0.
I20260812 14:50:07.290184 11246 leader_election.cc:302] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e, 23d2851107984eeda2a52e24feec9e3c; no voters:
I20260812 14:50:07.290463 11506 raft_consensus.cc:2781] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:50:07.290546 11506 raft_consensus.cc:482] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:50:07.290572 11506 raft_consensus.cc:3037] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:07.291337 11506 raft_consensus.cc:504] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.291652 11506 leader_election.cc:288] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [CANDIDATE]: Term 1 election: Requested vote from peers 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689), 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:07.291697 11125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d1d9b09d94fe29db2560452f7b482" candidate_uuid: "23d2851107984eeda2a52e24feec9e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2492c2247bda4c999231969d2a2d13d3"
I20260812 14:50:07.291702 10870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68d1d9b09d94fe29db2560452f7b482" candidate_uuid: "23d2851107984eeda2a52e24feec9e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
I20260812 14:50:07.291795 11125 raft_consensus.cc:3037] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:07.291812 10870 raft_consensus.cc:3037] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:07.292616 11125 raft_consensus.cc:2454] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d2851107984eeda2a52e24feec9e3c in term 1.
I20260812 14:50:07.292635 10870 raft_consensus.cc:2454] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d2851107984eeda2a52e24feec9e3c in term 1.
I20260812 14:50:07.292804 11246 leader_election.cc:302] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e, 23d2851107984eeda2a52e24feec9e3c; no voters:
I20260812 14:50:07.292936 11506 raft_consensus.cc:2781] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:50:07.293126 11506 raft_consensus.cc:686] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 1 LEADER]: Becoming Leader. State: Replica: 23d2851107984eeda2a52e24feec9e3c, State: Running, Role: LEADER
I20260812 14:50:07.293251 11506 consensus_queue.cc:237] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:07.294220 10602 catalog_manager.cc:5495] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 23d2851107984eeda2a52e24feec9e3c (127.13.146.131). New cstate: current_term: 1 leader_uuid: "23d2851107984eeda2a52e24feec9e3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } health_report { overall_health: UNKNOWN } } }
I20260812 14:50:07.328299 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.325682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.13.146.131:41815)
0812 14:50:07.325756 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:07.325758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:07.325769 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:50:07.325851 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:07.325852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:07.325872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:07.325950 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:50:07.326122 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.326595 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.327478 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:07.327674 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:07.327754 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:07.327912 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:50:07.328023 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:50:07.328159 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:07.328178 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 14:50:07.328200 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:50:07.330305 11125 raft_consensus.cc:1261] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Refusing update from remote peer 23d2851107984eeda2a52e24feec9e3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:50:07.330305 10870 raft_consensus.cc:1261] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Refusing update from remote peer 23d2851107984eeda2a52e24feec9e3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:50:07.330742 11513 consensus_queue.cc:1035] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:50:07.330899 11506 consensus_queue.cc:1035] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:50:07.333586 11518 mvcc.cc:205] Tried to move back new op lower bound from 7317693265221877760 to 7317693265073745920. Current Snapshot: MvccSnapshot[applied={T|T < 7317693265221877760}]
I20260812 14:50:07.336963 10598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:50:07.337070 10598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:50:07.337968 10598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Sending DeleteTablet for 3 replicas of tablet f68d1d9b09d94fe29db2560452f7b482
I20260812 14:50:07.338315 11328 tablet_service.cc:1514] Processing DeleteTablet for tablet f68d1d9b09d94fe29db2560452f7b482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:49560
I20260812 14:50:07.338332 11090 tablet_service.cc:1514] Processing DeleteTablet for tablet f68d1d9b09d94fe29db2560452f7b482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57040
I20260812 14:50:07.338598 11535 tablet_replica.cc:330] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: stopping tablet replica
I20260812 14:50:07.338613 10834 tablet_service.cc:1514] Processing DeleteTablet for tablet f68d1d9b09d94fe29db2560452f7b482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:45220
I20260812 14:50:07.338714 11535 raft_consensus.cc:2229] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:50:07.338841 11535 raft_consensus.cc:2258] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:50:07.338903 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 10715
I20260812 14:50:07.351573 11535 ts_tablet_manager.cc:1905] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:50:07.354180 11535 ts_tablet_manager.cc:1918] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:50:07.354307 11535 log.cc:1196] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/f68d1d9b09d94fe29db2560452f7b482
I20260812 14:50:07.354662 11535 ts_tablet_manager.cc:1939] T f68d1d9b09d94fe29db2560452f7b482 P 2492c2247bda4c999231969d2a2d13d3: Deleting consensus metadata
I20260812 14:50:07.371058 11534 tablet_replica.cc:330] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: stopping tablet replica
I20260812 14:50:07.371019 10579 catalog_manager.cc:4841] TS 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): tablet f68d1d9b09d94fe29db2560452f7b482 (table test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]) successfully deleted
I20260812 14:50:07.371233 11534 raft_consensus.cc:2229] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:50:07.371450 11534 raft_consensus.cc:2258] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:50:07.376543 11534 ts_tablet_manager.cc:1905] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:50:07.378821 11534 ts_tablet_manager.cc:1918] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:50:07.378927 11534 log.cc:1196] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/f68d1d9b09d94fe29db2560452f7b482
I20260812 14:50:07.379247 11534 ts_tablet_manager.cc:1939] T f68d1d9b09d94fe29db2560452f7b482 P 23d2851107984eeda2a52e24feec9e3c: Deleting consensus metadata
I20260812 14:50:07.379824 10574 catalog_manager.cc:4841] TS 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): tablet f68d1d9b09d94fe29db2560452f7b482 (table test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]) successfully deleted
W20260812 14:50:07.381996 10578 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:43673 (error 108)
I20260812 14:50:07.382318 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 10967
I20260812 14:50:07.385839 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:07.385676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.0.0.1:45230)
0812 14:50:07.385742 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:07.385794 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 14:50:07.385983 10578 catalog_manager.cc:4584] TS 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f68d1d9b09d94fe29db2560452f7b482: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43673: connect: Connection refused (error 111)
I20260812 14:50:07.393426 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 11217
I20260812 14:50:07.413395 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 10551
I20260812 14:50:27.430501 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--webserver_interface=127.13.146.190
--webserver_port=38097
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:41411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:27.517616 11542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:27.517871 11542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:27.517908 11542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:27.519058 11542 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:50:27.519127 11542 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:50:27.519148 11542 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:50:27.519169 11542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:50:27.519201 11542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:27.519227 11542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:27.519253 11542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:50:27.519276 11542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:50:27.520785 11542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:41411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=38097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:27.521095 11542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:27.521409 11542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:50:27.531694 11542 server_base.cc:1034] running on GCE node
W20260812 14:50:27.531888 11555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.531759 11552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.531762 11551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:27.532377 11542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:27.532752 11542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:27.533918 11542 hybrid_clock.cc:648] HybridClock initialized: now 1786546227533914 us; error 38 us; skew 500 ppm
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:27.541929 11542 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:50:27.542495 11542 webserver.cc:460] Webserver started at http://127.13.146.190:38097/ using document root <none> and password file <none>
I20260812 14:50:27.542753 11542 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:27.542848 11542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:27.544279 11542 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:27.545037 11564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.545270 11542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:27.545383 11542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "22ef993286194dda890a7cb7bfb3f61e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "f741967bf1c3bcf6e1ffb49f3444ba13"
server_key_iv: "cb0012025f425bdf696fe23fd867da8b"
server_key_version: "encryptionkey@0"
I20260812 14:50:27.545845 11542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:27.559927 11542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:27.562364 11542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:27.562538 11542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:27.567106 11542 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:41411
I20260812 14:50:27.567147 11659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:41411 every 8 connection(s)
I20260812 14:50:27.567595 11542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:50:27.569337 11660 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:50:27.570137 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap starting.
I20260812 14:50:27.577698 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 11542
I20260812 14:50:27.578016 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:43673
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=36689
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:50:27.580731 11660 log.cc:824] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:27.583079 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:50:27.583459 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap complete.
I20260812 14:50:27.586304 11660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:27.586606 11660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Initialized, Role: FOLLOWER
I20260812 14:50:27.586853 11660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:27.586969 11660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:50:27.587039 11660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:50:27.587100 11660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:50:27.588280 11660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:27.588421 11660 leader_election.cc:302] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22ef993286194dda890a7cb7bfb3f61e; no voters:
I20260812 14:50:27.588647 11660 leader_election.cc:288] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:50:27.588721 11666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:50:27.588904 11666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 LEADER]: Becoming Leader. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Running, Role: LEADER
I20260812 14:50:27.588953 11660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:50:27.589107 11666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:27.589494 11667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:27.589521 11668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22ef993286194dda890a7cb7bfb3f61e. Latest consensus state: current_term: 2 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:27.589671 11667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
I20260812 14:50:27.589690 11668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
I20260812 14:50:27.590476 11676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:50:27.591477 11676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:50:27.591593 11676 catalog_manager.cc:1259] Loaded cluster ID: 705fd7227ab3455692afdf4570ea2bac
I20260812 14:50:27.591667 11676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:50:27.592743 11676 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:50:27.592960 11676 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Loaded TSK: 0
I20260812 14:50:27.598920 11676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Generated new TSK 1
I20260812 14:50:27.599032 11676 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:50:27.675213 11662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:27.675444 11662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:27.675485 11662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:27.676620 11662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:27.676689 11662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:27.676734 11662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:50:27.678303 11662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:43673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=36689
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:27.678617 11662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:27.678859 11662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:27.684979 11710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.685038 11705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.685017 11701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:27.685318 11662 server_base.cc:1034] running on GCE node
I20260812 14:50:27.685554 11662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:27.685838 11662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:27.687032 11662 hybrid_clock.cc:648] HybridClock initialized: now 1786546227687008 us; error 38 us; skew 500 ppm
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:27.690208 11662 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:50:27.690701 11662 webserver.cc:460] Webserver started at http://127.13.146.129:36689/ using document root <none> and password file <none>
I20260812 14:50:27.690979 11662 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:27.691075 11662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:27.692556 11662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:50:27.693311 11721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.693496 11662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:27.693598 11662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "c01c9d2f1a26d355e187640cd422cced"
server_key_iv: "33edff7061fe47a3f677a5ab3ada6a0b"
server_key_version: "encryptionkey@0"
I20260812 14:50:27.693991 11662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:27.710948 11662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:27.713474 11662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:27.713653 11662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:27.713929 11662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:27.714475 11734 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:50:27.716105 11662 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:50:27.716168 11662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:50:27.716195 11662 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:50:27.716898 11662 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:50:27.716945 11662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:50:27.716998 11734 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap starting.
I20260812 14:50:27.725234 11662 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:43673
I20260812 14:50:27.725473 11910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:43673 every 8 connection(s)
I20260812 14:50:27.725793 11662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:50:27.729094 11734 log.cc:824] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:27.731846 11734 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:50:27.732194 11734 tablet_bootstrap.cc:492] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap complete.
I20260812 14:50:27.733528 11734 ts_tablet_manager.cc:1397] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.007s
I20260812 14:50:27.734853 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 11662
I20260812 14:50:27.734717 11734 raft_consensus.cc:348] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:27.735072 11734 raft_consensus.cc:729] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e6ea22be8b490d91bcb2153d06cd7e, State: Initialized, Role: FOLLOWER
I20260812 14:50:27.735173 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:42689
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=45109
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:50:27.735272 11734 consensus_queue.cc:260] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:27.735558 11734 ts_tablet_manager.cc:1428] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:27.742314 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.726647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:44313)
0812 14:50:27.729385 (+ 2738us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:27.729435 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:27.730091 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:27.730820 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:27.730824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:27.731186 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:27.731477 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.731485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.733261 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.733262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.734159 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.734162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.735090 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.735092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.735510 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.736065 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:27.736076 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:27.737741 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:27.740448 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.740456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.740458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.740727 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.741095 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.741096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.741096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.741275 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.741801 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:27.741807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:27.741928 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 14:50:27.742020 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:50:27.742159 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2673,"thread_start_us":145,"threads_started":1}
I20260812 14:50:27.742538 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.726729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:44313 (local address 127.13.146.190:41411)
0812 14:50:27.730314 (+ 3585us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:27.730323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:27.730364 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:50:27.730505 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:27.730512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:27.730568 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:27.730729 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:27.731627 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.733088 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.734260 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.734866 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.734979 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.737851 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:27.737855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:27.737856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:27.737867 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:27.740343 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.740836 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.740840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.740841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.740970 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.741373 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.741374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.741375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.741721 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:27.741821 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:27.742128 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 14:50:27.742289 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3526,"thread_start_us":98,"threads_started":1}
I20260812 14:50:27.743352 11916 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:27.743487 11916 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:27.743708 11916 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:27.744556 11600 ts_manager.cc:194] Registered new tserver with Master: 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:27.745941 11600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:44313
I20260812 14:50:27.746963 11916 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.129@KRBTEST.COM
I20260812 14:50:27.754911 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.745282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.0.0.1:37658)
0812 14:50:27.745552 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:27.745597 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:27.745656 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:27.745913 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:27.745919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:27.746373 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:27.746436 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.746439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.747813 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.747814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.748576 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.748579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.749272 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.749273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.749352 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.749947 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:27.749953 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:27.751610 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:27.753700 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.753707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.753709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.753927 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.754176 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.754177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.754178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.754244 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.754588 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:27.754595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:27.754641 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:50:27.754712 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:50:27.754811 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260812 14:50:27.755013 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.745345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.13.146.129:43673)
0812 14:50:27.745764 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:27.745767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:27.745775 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:50:27.745802 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:27.745804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:27.745816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:27.745908 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:27.746545 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.747717 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.748657 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.749130 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.749233 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.751703 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:27.751710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:27.751711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:27.751718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:27.753613 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.754010 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.754012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.754013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.754079 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.754315 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.754315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.754316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.754491 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:27.754574 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:27.754773 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 14:50:27.754884 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":391,"thread_start_us":83,"threads_started":1}
I20260812 14:50:27.755445 11803 tablet_service.cc:1514] Processing DeleteTablet for tablet f68d1d9b09d94fe29db2560452f7b482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:07 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:37658
I20260812 14:50:27.756031 11929 tablet_replica.cc:330] stopping tablet replica
I20260812 14:50:27.756166 11929 raft_consensus.cc:2229] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:50:27.756251 11929 raft_consensus.cc:2258] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:50:27.756565 11929 ts_tablet_manager.cc:1905] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:50:27.758671 11929 ts_tablet_manager.cc:1918] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:50:27.758764 11929 log.cc:1196] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal/wals/f68d1d9b09d94fe29db2560452f7b482
I20260812 14:50:27.758986 11929 ts_tablet_manager.cc:1939] T f68d1d9b09d94fe29db2560452f7b482 P 07e6ea22be8b490d91bcb2153d06cd7e: Deleting consensus metadata
I20260812 14:50:27.759485 11569 catalog_manager.cc:4841] TS 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673): tablet f68d1d9b09d94fe29db2560452f7b482 (table test-table [id=7b7f51cf23ee47cba946ce5b28960dbf]) successfully deleted
W20260812 14:50:27.825841 11924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:27.826056 11924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:27.826079 11924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:27.827235 11924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:27.827292 11924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:27.827327 11924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:50:27.828824 11924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=45109
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:27.829027 11924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:27.829236 11924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:27.834548 11941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.834561 11945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.834745 11940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:27.834976 11924 server_base.cc:1034] running on GCE node
I20260812 14:50:27.835179 11924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:27.835407 11924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:27.836588 11924 hybrid_clock.cc:648] HybridClock initialized: now 1786546227836553 us; error 74 us; skew 500 ppm
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:27.839455 11924 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:50:27.839934 11924 webserver.cc:460] Webserver started at http://127.13.146.130:45109/ using document root <none> and password file <none>
I20260812 14:50:27.840180 11924 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:27.840262 11924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:27.841971 11924 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:50:27.842747 11963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.842981 11924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:27.843099 11924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "2492c2247bda4c999231969d2a2d13d3"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "e011ce707b91f1a4c28c7d961802a5d7"
server_key_iv: "838e7d0ab3beadf725fcede5f148766e"
server_key_version: "encryptionkey@0"
I20260812 14:50:27.843487 11924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:27.852506 11924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:27.854569 11924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:27.854763 11924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:27.855005 11924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:27.855321 11924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:27.855377 11924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.855422 11924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:27.855446 11924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.861575 11924 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:42689
I20260812 14:50:27.861630 12152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:42689 every 8 connection(s)
I20260812 14:50:27.861967 11924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:27.871109 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.862756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:45575 (local address 127.13.146.190:41411)
0812 14:50:27.862849 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:27.862851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:27.863059 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:50:27.863571 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:27.863572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:27.863587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:27.863646 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:27.864653 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.865169 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.866172 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.866375 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.866433 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.868506 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:27.868507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:27.868508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:27.868514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:27.869982 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.870333 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.870334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.870334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.870387 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:27.870597 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:27.870597 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:27.870598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:27.870697 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:27.870747 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:27.870973 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 14:50:27.871042 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 14:50:27.871308 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:27.862706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:45575)
0812 14:50:27.862893 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:27.862936 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:27.863471 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:27.863742 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:27.863745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:27.864353 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:27.864549 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.864555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.865293 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.865294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.866089 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:27.866091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:27.866455 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:27.866456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:27.866557 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:27.867036 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:27.867044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:27.868419 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:27.870055 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.870060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.870062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.870268 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.870444 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:27.870445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:27.870445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:27.870540 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:27.870753 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:27.870757 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:27.870866 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 14:50:27.870947 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:50:27.871060 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260812 14:50:27.871852 12154 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:27.871965 12154 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:27.872149 12154 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:27.872262 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 11924
I20260812 14:50:27.872615 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:41815
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=39361
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:50:27.872641 11600 ts_manager.cc:194] Registered new tserver with Master: 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
I20260812 14:50:27.873106 11600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:45575
W20260812 14:50:27.959019 12164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:27.959228 12164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:27.959252 12164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:27.960318 12164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:27.960354 12164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:27.960379 12164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:50:27.961824 12164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:41815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=39361
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:27.962028 12164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:27.962204 12164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:27.967448 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:27.967422 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:27.967604 12164 server_base.cc:1034] running on GCE node
W20260812 14:50:27.967437 12179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:27.967914 12164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:27.968148 12164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:27.969295 12164 hybrid_clock.cc:648] HybridClock initialized: now 1786546227969283 us; error 31 us; skew 500 ppm
Aug 12 14:50:27 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:27.971885 12164 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:50:27.972373 12164 webserver.cc:460] Webserver started at http://127.13.146.131:39361/ using document root <none> and password file <none>
I20260812 14:50:27.972618 12164 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:27.972705 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:27.974172 12164 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:27.974807 12191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.975063 12164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:27.975200 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "23d2851107984eeda2a52e24feec9e3c"
format_stamp: "Formatted at 2026-08-12 14:50:07 on dist-test-slave-8hhm"
server_key: "f11a29c31ba0678c8038a16f48ff76f6"
server_key_iv: "65e0222473c77e3481f9514ec00d31ba"
server_key_version: "encryptionkey@0"
I20260812 14:50:27.975612 12164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:27.991549 12164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:27.993728 12164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:27.993914 12164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:27.994181 12164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:27.994529 12164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:27.994591 12164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:27.994637 12164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:27.994660 12164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:28.001554 12164 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41815
I20260812 14:50:28.001600 12385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41815 every 8 connection(s)
I20260812 14:50:28.002028 12164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:50:28 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546227, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:50:28.009013 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 12164
I20260812 14:50:28.012913 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.002852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:51581 (local address 127.13.146.190:41411)
0812 14:50:28.002925 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.002927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.003152 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:50:28.003699 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.003700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.003710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.003770 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:28.004789 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.005336 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.006471 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.006658 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.006708 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.009346 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:28.009348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:28.009348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:28.009358 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:28.011317 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.011808 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.011810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.011810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.011890 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.012182 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.012183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.012184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.012317 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:28.012402 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.012709 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.012807 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:50:28.013192 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.002828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:51581)
0812 14:50:28.003009 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.003057 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:28.003605 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:28.003833 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:28.003836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:28.004475 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:28.004699 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.004706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.005466 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.005468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.006384 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.006386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.006739 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.006740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.006851 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.007383 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:28.007392 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:28.009205 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:28.011410 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:28.011417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:28.011419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:28.011673 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:28.011969 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:28.011970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:28.011970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:28.012100 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:28.012393 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:28.012401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:28.012560 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 14:50:28.012683 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 14:50:28.012865 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260812 14:50:28.013744 12387 heartbeater.cc:347] Connected to a master server at 127.13.146.190:41411
I20260812 14:50:28.013872 12387 heartbeater.cc:464] Registering TS with master...
I20260812 14:50:28.014089 12387 heartbeater.cc:511] Master 127.13.146.190:41411 requested a full tablet report, sending...
I20260812 14:50:28.014567 11600 ts_manager.cc:194] Registered new tserver with Master: 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815)
I20260812 14:50:28.014943 11600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:51581
I20260812 14:50:28.016968 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.009515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.13.146.190:41411)
0812 14:50:28.009705 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.009707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.009750 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:50:28.009838 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.009839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.009850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.009938 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:28.010818 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.011564 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.012360 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.012563 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.012642 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.014158 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:28.014159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:28.014160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:28.014167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:28.015955 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.016315 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.016316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.016316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.016374 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.016551 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.016552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.016552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.016655 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:28.016779 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.016833 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.016901 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
I20260812 14:50:28.017851 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:50:28.020512 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.018242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.13.146.190:41411)
0812 14:50:28.018286 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.018290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.018427 (+ 137us) server_negotiation.cc:407] Connection header received
0812 14:50:28.018465 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.018466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.018498 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.018563 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:50:28.018676 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.019186 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.019881 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.020106 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.020160 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.020276 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:50:28.020316 (+ 40us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 14:50:28.020430 (+ 114us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33102: token expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:50:28.020574 12393 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33102: token expired
W20260812 14:50:28.020749 11465 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.146.190:41411)
I20260812 14:50:28.027048 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.021058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.13.146.190:41411)
0812 14:50:28.021121 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.021123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.021130 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:50:28.021148 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.021149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.021165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.021248 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:28.021723 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.022186 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.022887 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.023067 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.023157 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.024261 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:28.024263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:28.024263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:28.024270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:28.025783 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.026155 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.026156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.026157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.026222 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:28.026433 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:28.026434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:28.026434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:28.026542 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:28.026650 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.026829 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.026932 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:50:28.028299 11600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:50:28.030026 11600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:50:28.032249 11803 tablet_service.cc:1467] Processing CreateTablet for tablet 2543ec6ca246462a864b04b80c00b322 (DEFAULT_TABLE table=test-table [id=00f5af47c2994f188b807aa63d457f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:28.032480 11803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2543ec6ca246462a864b04b80c00b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:28.033902 11734 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap starting.
I20260812 14:50:28.034257 11734 tablet_bootstrap.cc:654] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:28.035413 11734 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: No bootstrap required, opened a new log
I20260812 14:50:28.035470 11734 ts_tablet_manager.cc:1397] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 14:50:28.035832 11734 raft_consensus.cc:348] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.035918 11734 raft_consensus.cc:374] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:28.035962 11734 raft_consensus.cc:729] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e6ea22be8b490d91bcb2153d06cd7e, State: Initialized, Role: FOLLOWER
I20260812 14:50:28.036023 11734 consensus_queue.cc:260] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.036116 11734 ts_tablet_manager.cc:1428] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:50:28.036862 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.031982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:41815 (local address 127.0.0.1:36058)
0812 14:50:28.032019 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.032032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:28.032069 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:28.032514 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:28.032515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:28.032517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:28.032566 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.032568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.033670 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.033671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.034716 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.034719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.036625 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.036627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.036710 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.036727 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 14:50:28.036775 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:50:28.036811 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 14:50:28.036926 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.032136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.13.146.131:41815)
0812 14:50:28.032348 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.032353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.032364 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:50:28.032404 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.032407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.032423 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.032509 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:28.032626 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.033578 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.034812 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.036526 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.036624 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.036690 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.036803 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.036842 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":134,"threads_started":1}
I20260812 14:50:28.037685 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.031982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42689 (local address 127.0.0.1:39232)
0812 14:50:28.033952 (+ 1970us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.033964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:28.034020 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:28.034257 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:28.034258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:28.034260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:28.034382 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.034384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.035322 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.035323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.036383 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.036386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.037435 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.037435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.037481 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.037484 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:50:28.037557 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:50:28.037600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1952,"mutex_wait_us":15,"thread_start_us":115,"threads_started":1}
I20260812 14:50:28.037761 12272 tablet_service.cc:1467] Processing CreateTablet for tablet 2543ec6ca246462a864b04b80c00b322 (DEFAULT_TABLE table=test-table [id=00f5af47c2994f188b807aa63d457f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:28.037935 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.032267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.13.146.130:42689)
0812 14:50:28.032476 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.032480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.034080 (+ 1600us) server_negotiation.cc:407] Connection header received
0812 14:50:28.034126 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.034128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.034144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.034248 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:28.034472 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.035251 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.036487 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.037277 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.037344 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.037396 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.037815 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.037858 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
I20260812 14:50:28.038215 12272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2543ec6ca246462a864b04b80c00b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:28.038512 12047 tablet_service.cc:1467] Processing CreateTablet for tablet 2543ec6ca246462a864b04b80c00b322 (DEFAULT_TABLE table=test-table [id=00f5af47c2994f188b807aa63d457f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:50:28.038858 12047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2543ec6ca246462a864b04b80c00b322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:50:28.041252 12406 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Bootstrap starting.
I20260812 14:50:28.041616 12408 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Bootstrap starting.
I20260812 14:50:28.042011 12406 tablet_bootstrap.cc:654] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:28.042371 12408 tablet_bootstrap.cc:654] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Neither blocks nor log segments found. Creating new log.
I20260812 14:50:28.042645 12406 log.cc:824] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:28.042994 12408 log.cc:824] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:28.043493 12406 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: No bootstrap required, opened a new log
I20260812 14:50:28.043573 12406 ts_tablet_manager.cc:1397] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:50:28.043774 12408 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: No bootstrap required, opened a new log
I20260812 14:50:28.043841 12408 ts_tablet_manager.cc:1397] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:50:28.045792 12408 raft_consensus.cc:348] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.045770 12406 raft_consensus.cc:348] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.045966 12408 raft_consensus.cc:374] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:28.045966 12406 raft_consensus.cc:374] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:50:28.046006 12408 raft_consensus.cc:729] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2492c2247bda4c999231969d2a2d13d3, State: Initialized, Role: FOLLOWER
I20260812 14:50:28.046006 12406 raft_consensus.cc:729] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d2851107984eeda2a52e24feec9e3c, State: Initialized, Role: FOLLOWER
I20260812 14:50:28.046136 12406 consensus_queue.cc:260] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.046151 12408 consensus_queue.cc:260] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.046443 12408 ts_tablet_manager.cc:1428] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:50:28.046482 12154 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
I20260812 14:50:28.046473 12387 heartbeater.cc:503] Master 127.13.146.190:41411 was elected leader, sending a full tablet report...
I20260812 14:50:28.046713 12406 ts_tablet_manager.cc:1428] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:50:28.064534 12413 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:28.064707 12413 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.065132 12413 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:28.068979 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.065417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:58943 (local address 127.13.146.131:41815)
0812 14:50:28.065472 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.065474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.065487 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:50:28.065554 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.065555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.065563 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.065641 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:28.065818 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.066585 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068010 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.068588 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068674 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.068683 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.068865 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.068903 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:50:28.068979 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.065440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:33897 (local address 127.13.146.129:43673)
0812 14:50:28.065498 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.065500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.065625 (+ 125us) server_negotiation.cc:407] Connection header received
0812 14:50:28.065718 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.065719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.065741 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.065821 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:50:28.065974 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.066721 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068014 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.068620 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068694 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.068778 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.068863 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.068904 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:50:28.069003 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.065326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:41815 (local address 127.13.146.130:58943)
0812 14:50:28.065419 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.065432 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:28.065481 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:28.065666 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:28.065667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:28.065669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:28.065745 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.065747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.066673 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.066674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.067911 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.067915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.068702 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.068784 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.068786 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:50:28.068843 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:50:28.068900 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":22}
I20260812 14:50:28.069010 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.065332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.13.146.130:33897)
0812 14:50:28.065542 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.065551 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:28.065607 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:28.065844 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:28.065845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:28.065846 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:50:28.065900 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.065900 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.066808 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.066808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.067911 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:28.067915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.068702 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.068703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:28.068781 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.068785 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:50:28.068843 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:50:28.068900 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":17,"thread_start_us":116,"threads_started":1}
I20260812 14:50:28.069387 12317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2543ec6ca246462a864b04b80c00b322" candidate_uuid: "2492c2247bda4c999231969d2a2d13d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d2851107984eeda2a52e24feec9e3c" is_pre_election: true
I20260812 14:50:28.069531 12317 raft_consensus.cc:2454] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2492c2247bda4c999231969d2a2d13d3 in term 0.
I20260812 14:50:28.069526 11835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2543ec6ca246462a864b04b80c00b322" candidate_uuid: "2492c2247bda4c999231969d2a2d13d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" is_pre_election: true
I20260812 14:50:28.069720 11835 raft_consensus.cc:2454] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2492c2247bda4c999231969d2a2d13d3 in term 0.
I20260812 14:50:28.069787 11966 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3; no voters:
I20260812 14:50:28.069936 12413 raft_consensus.cc:2781] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:50:28.069996 12413 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:50:28.070019 12413 raft_consensus.cc:3037] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:28.070724 12413 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.070869 12413 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [CANDIDATE]: Term 1 election: Requested vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673)
I20260812 14:50:28.071017 12317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2543ec6ca246462a864b04b80c00b322" candidate_uuid: "2492c2247bda4c999231969d2a2d13d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d2851107984eeda2a52e24feec9e3c"
I20260812 14:50:28.071014 11835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2543ec6ca246462a864b04b80c00b322" candidate_uuid: "2492c2247bda4c999231969d2a2d13d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
I20260812 14:50:28.071095 11835 raft_consensus.cc:3037] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:28.071118 12317 raft_consensus.cc:3037] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:50:28.071889 12317 raft_consensus.cc:2454] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2492c2247bda4c999231969d2a2d13d3 in term 1.
I20260812 14:50:28.071895 11835 raft_consensus.cc:2454] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2492c2247bda4c999231969d2a2d13d3 in term 1.
I20260812 14:50:28.072108 11967 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e, 2492c2247bda4c999231969d2a2d13d3; no voters:
I20260812 14:50:28.072222 12413 raft_consensus.cc:2781] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:50:28.072378 12413 raft_consensus.cc:686] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2492c2247bda4c999231969d2a2d13d3, State: Running, Role: LEADER
I20260812 14:50:28.072453 12413 consensus_queue.cc:237] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:28.073390 11599 catalog_manager.cc:5495] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130). New cstate: current_term: 1 leader_uuid: "2492c2247bda4c999231969d2a2d13d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } health_report { overall_health: UNKNOWN } } }
I20260812 14:50:28.082091 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.079615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.13.146.130:42689)
0812 14:50:28.079677 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:28.079678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:28.079707 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:50:28.079796 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:28.079797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:28.079815 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:28.079874 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:50:28.080028 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.080480 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.081318 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:28.081557 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:28.081635 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:28.081737 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:50:28.081831 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:50:28.081892 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:28.081924 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:50:28.081940 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:50:28.083956 12317 raft_consensus.cc:1261] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Refusing update from remote peer 2492c2247bda4c999231969d2a2d13d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:50:28.083969 11835 raft_consensus.cc:1261] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Refusing update from remote peer 2492c2247bda4c999231969d2a2d13d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:50:28.084365 12420 consensus_queue.cc:1035] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:50:28.084528 12413 consensus_queue.cc:1035] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:50:28.086634 12426 mvcc.cc:205] Tried to move back new op lower bound from 7317693350229155840 to 7317693350185529344. Current Snapshot: MvccSnapshot[applied={T|T < 7317693350229155840}]
I20260812 14:50:28.086735 12427 mvcc.cc:205] Tried to move back new op lower bound from 7317693350229155840 to 7317693350185529344. Current Snapshot: MvccSnapshot[applied={T|T < 7317693350229155840}]
I20260812 14:50:28.087816 12434 mvcc.cc:205] Tried to move back new op lower bound from 7317693350229155840 to 7317693350185529344. Current Snapshot: MvccSnapshot[applied={T|T < 7317693350229155840}]
I20260812 14:50:28.089177 11600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:50:28.089291 11600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:50:28.090114 11600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Sending DeleteTablet for 3 replicas of tablet 2543ec6ca246462a864b04b80c00b322
I20260812 14:50:28.090513 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 11662
I20260812 14:50:28.090538 12272 tablet_service.cc:1514] Processing DeleteTablet for tablet 2543ec6ca246462a864b04b80c00b322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:28 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:36058
I20260812 14:50:28.090566 12047 tablet_service.cc:1514] Processing DeleteTablet for tablet 2543ec6ca246462a864b04b80c00b322 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:50:28 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:39232
I20260812 14:50:28.097364 12444 tablet_replica.cc:330] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: stopping tablet replica
I20260812 14:50:28.097546 12442 tablet_replica.cc:330] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: stopping tablet replica
I20260812 14:50:28.097730 12442 raft_consensus.cc:2229] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:50:28.097886 12444 raft_consensus.cc:2229] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:50:28.098080 12442 raft_consensus.cc:2258] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:50:28.098253 12444 raft_consensus.cc:2258] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:50:28.100234 12444 ts_tablet_manager.cc:1905] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:50:28.101600 12442 ts_tablet_manager.cc:1905] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:50:28.102236 12444 ts_tablet_manager.cc:1918] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:50:28.102324 12444 log.cc:1196] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/2543ec6ca246462a864b04b80c00b322
I20260812 14:50:28.102666 12444 ts_tablet_manager.cc:1939] T 2543ec6ca246462a864b04b80c00b322 P 2492c2247bda4c999231969d2a2d13d3: Deleting consensus metadata
I20260812 14:50:28.103185 11571 catalog_manager.cc:4841] TS 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): tablet 2543ec6ca246462a864b04b80c00b322 (table test-table [id=00f5af47c2994f188b807aa63d457f3c]) successfully deleted
I20260812 14:50:28.103968 12442 ts_tablet_manager.cc:1918] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:50:28.104050 12442 log.cc:1196] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/2543ec6ca246462a864b04b80c00b322
I20260812 14:50:28.104327 12442 ts_tablet_manager.cc:1939] T 2543ec6ca246462a864b04b80c00b322 P 23d2851107984eeda2a52e24feec9e3c: Deleting consensus metadata
I20260812 14:50:28.104807 11568 catalog_manager.cc:4841] TS 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): tablet 2543ec6ca246462a864b04b80c00b322 (table test-table [id=00f5af47c2994f188b807aa63d457f3c]) successfully deleted
W20260812 14:50:28.105614 11569 connection.cc:535] client connection to 127.13.146.129:43673 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:50:28.105762 11569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:50:28.106045 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 11924
I20260812 14:50:28.106823 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:28.106167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43673 (local address 127.0.0.1:37670)
0812 14:50:28.106737 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:28.106781 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548}
W20260812 14:50:28.111183 11569 catalog_manager.cc:4584] TS 07e6ea22be8b490d91bcb2153d06cd7e (127.13.146.129:43673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2543ec6ca246462a864b04b80c00b322: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43673: connect: Connection refused (error 111)
I20260812 14:50:28.122519 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 12164
I20260812 14:50:28.141732 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 11542
I20260812 14:50:48.154842 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--webserver_interface=127.13.146.190
--webserver_port=38097
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:41411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:48.240777 12449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:48.240988 12449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:48.241010 12449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:48.242195 12449 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:50:48.242237 12449 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:50:48.242250 12449 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:50:48.242298 12449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:50:48.242326 12449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:48.242352 12449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:48.242383 12449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:50:48.242412 12449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:50:48.243884 12449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:41411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=38097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:48.244148 12449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:48.244401 12449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:48.249794 12456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:48.249993 12449 server_base.cc:1034] running on GCE node
W20260812 14:50:48.249847 12458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.250098 12461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:48.250422 12449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:48.250912 12449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:48.252141 12449 hybrid_clock.cc:648] HybridClock initialized: now 1786546248252142 us; error 45 us; skew 500 ppm
Aug 12 14:50:48 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:48.260231 12449 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:50:48.260773 12449 webserver.cc:460] Webserver started at http://127.13.146.190:38097/ using document root <none> and password file <none>
I20260812 14:50:48.261011 12449 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:48.261080 12449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:48.262604 12449 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:48.263362 12474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.263545 12449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:50:48.263654 12449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "22ef993286194dda890a7cb7bfb3f61e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "f741967bf1c3bcf6e1ffb49f3444ba13"
server_key_iv: "cb0012025f425bdf696fe23fd867da8b"
server_key_version: "encryptionkey@0"
I20260812 14:50:48.264055 12449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:48.310858 12449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:48.313607 12449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:48.313802 12449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:48.318269 12449 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:41411
I20260812 14:50:48.318313 12566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:41411 every 8 connection(s)
I20260812 14:50:48.318727 12449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:50:48.320276 12568 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:50:48.321010 12568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap starting.
I20260812 14:50:48.321908 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 12449
I20260812 14:50:48.322230 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:43673
--local_ip_for_outbound_sockets=127.13.146.129
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=36689
--webserver_interface=127.13.146.129
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:50:48.331723 12568 log.cc:824] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:48.334263 12568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:50:48.334595 12568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Bootstrap complete.
I20260812 14:50:48.337065 12568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:48.337369 12568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Initialized, Role: FOLLOWER
I20260812 14:50:48.337550 12568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:48.337610 12568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:50:48.337674 12568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:50:48.337728 12568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:50:48.338730 12568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:48.338855 12568 leader_election.cc:302] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22ef993286194dda890a7cb7bfb3f61e; no voters:
I20260812 14:50:48.339068 12568 leader_election.cc:288] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:50:48.339120 12573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:50:48.339258 12573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [term 3 LEADER]: Becoming Leader. State: Replica: 22ef993286194dda890a7cb7bfb3f61e, State: Running, Role: LEADER
I20260812 14:50:48.339359 12568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:50:48.339427 12573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } }
I20260812 14:50:48.339708 12573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22ef993286194dda890a7cb7bfb3f61e. Latest consensus state: current_term: 3 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:48.339800 12573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
I20260812 14:50:48.339715 12575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "22ef993286194dda890a7cb7bfb3f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22ef993286194dda890a7cb7bfb3f61e" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41411 } } }
I20260812 14:50:48.340005 12575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e [sys.catalog]: This master's current role is: LEADER
I20260812 14:50:48.340411 12587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:50:48.341298 12587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:50:48.341470 12587 catalog_manager.cc:1259] Loaded cluster ID: 705fd7227ab3455692afdf4570ea2bac
I20260812 14:50:48.341665 12587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:50:48.342885 12587 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:50:48.343052 12587 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Loaded TSK: 0
I20260812 14:50:48.343091 12587 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 22ef993286194dda890a7cb7bfb3f61e: Loaded TSK: 1
F20260812 14:50:48.343333 12587 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f19adc009ad google::LogMessage::Fail() at ??:0
@ 0x7f19adc0388a google::LogMessage::SendToLog() at ??:0
@ 0x7f19adc004de google::LogMessage::Flush() at ??:0
@ 0x7f19adc01a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f19b036d104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f19b036ad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f19b5b5a1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f19b5b5a48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f19b5b5a594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f19b5b20c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f19b5b2aad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f19aface0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f19afac3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f19b09276db start_thread at ??:0
@ 0x7f19ac6fe61f clone at ??:0
W20260812 14:50:48.427871 12570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:48.428133 12570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:48.428180 12570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:48.429388 12570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:48.429443 12570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:48.429487 12570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:50:48.430951 12570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:43673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=36689
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:48.431218 12570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:48.431449 12570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:48.436668 12652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.436704 12653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.436720 12657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:48.437132 12570 server_base.cc:1034] running on GCE node
I20260812 14:50:48.437415 12570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:48.437737 12570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:48.438930 12570 hybrid_clock.cc:648] HybridClock initialized: now 1786546248438914 us; error 28 us; skew 500 ppm
Aug 12 14:50:48 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:48.442142 12570 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:50:48.442732 12570 webserver.cc:460] Webserver started at http://127.13.146.129:36689/ using document root <none> and password file <none>
I20260812 14:50:48.443034 12570 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:48.443112 12570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:48.444921 12570 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:50:48.445755 12667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.445945 12570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:50:48.446026 12570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "07e6ea22be8b490d91bcb2153d06cd7e"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "c01c9d2f1a26d355e187640cd422cced"
server_key_iv: "33edff7061fe47a3f677a5ab3ada6a0b"
server_key_version: "encryptionkey@0"
I20260812 14:50:48.446373 12570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:48.463169 12570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:48.465687 12570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:48.465854 12570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:48.466122 12570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:48.466586 12679 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:50:48.468724 12570 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:50:48.468783 12570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:50:48.468845 12570 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:50:48.469624 12570 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:50:48.469678 12570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:48.469767 12679 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap starting.
I20260812 14:50:48.477309 12570 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:43673
I20260812 14:50:48.477392 12879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:43673 every 8 connection(s)
I20260812 14:50:48.477783 12570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:50:48.478761 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 12570
I20260812 14:50:48.479275 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:42689
--local_ip_for_outbound_sockets=127.13.146.130
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=45109
--webserver_interface=127.13.146.130
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:50:48.482345 12679 log.cc:824] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:50:48.484719 12679 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:50:48.485013 12679 tablet_bootstrap.cc:492] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Bootstrap complete.
I20260812 14:50:48.486066 12679 ts_tablet_manager.cc:1397] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.014s
I20260812 14:50:48.487138 12679 raft_consensus.cc:348] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:48.487409 12679 raft_consensus.cc:729] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e6ea22be8b490d91bcb2153d06cd7e, State: Initialized, Role: FOLLOWER
I20260812 14:50:48.487612 12679 consensus_queue.cc:260] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:48.487926 12679 ts_tablet_manager.cc:1428] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260812 14:50:48.574816 12883 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:50:48.478668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:33829)
0812 14:50:48.478890 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.478945 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:48.479712 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:48.574648 (+ 94936us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
W20260812 14:50:48.575119 12880 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:41411 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:41411: Client connection negotiation failed: client connection to 127.13.146.190:41411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:50:48.575487 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.575371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:47817)
0812 14:50:48.575424 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.575467 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:50:48.575897 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.575813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:60745)
0812 14:50:48.575858 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.575882 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:50:48.576040 12880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:50:48.585378 12885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:48.585585 12885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:48.585610 12885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:48.586735 12885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:48.586773 12885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:48.586800 12885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:50:48.588253 12885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=45109
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:48.588446 12885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:48.588625 12885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:48.594409 12903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.594404 12900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.594427 12905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:48.594831 12885 server_base.cc:1034] running on GCE node
I20260812 14:50:48.595104 12885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:48.595360 12885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:48.596540 12885 hybrid_clock.cc:648] HybridClock initialized: now 1786546248596521 us; error 46 us; skew 500 ppm
Aug 12 14:50:48 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:48.599449 12885 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:50:48.599910 12885 webserver.cc:460] Webserver started at http://127.13.146.130:45109/ using document root <none> and password file <none>
I20260812 14:50:48.600139 12885 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:48.600207 12885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:48.601676 12885 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:50:48.602317 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.602487 12885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:50:48.602600 12885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "2492c2247bda4c999231969d2a2d13d3"
format_stamp: "Formatted at 2026-08-12 14:50:06 on dist-test-slave-8hhm"
server_key: "e011ce707b91f1a4c28c7d961802a5d7"
server_key_iv: "838e7d0ab3beadf725fcede5f148766e"
server_key_version: "encryptionkey@0"
I20260812 14:50:48.602960 12885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:48.621491 12885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:48.623934 12885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:48.624104 12885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:48.624364 12885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:48.624689 12885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:48.624744 12885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.624781 12885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:48.624825 12885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.631174 12885 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:42689
I20260812 14:50:48.631222 13121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:42689 every 8 connection(s)
I20260812 14:50:48.631579 12885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:50:48.632709 13130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.632325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:41317)
0812 14:50:48.632485 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.632564 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
W20260812 14:50:48.633304 13125 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:41411 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:41411: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
I20260812 14:50:48.633605 13130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.633494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:45745)
0812 14:50:48.633549 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.633583 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:50:48.633865 13130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.633793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:35609)
0812 14:50:48.633830 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.633851 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:50:48.633970 13125 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:50:48.638962 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 12885
I20260812 14:50:48.639287 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:41815
--local_ip_for_outbound_sockets=127.13.146.131
--tserver_master_addrs=127.13.146.190:41411
--webserver_port=39361
--webserver_interface=127.13.146.131
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:50:48.724699 13131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:50:48.724911 13131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:50:48.724934 13131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:50:48.726173 13131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:50:48.726224 13131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:50:48.726253 13131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:50:48.727859 13131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:41815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=39361
--tserver_master_addrs=127.13.146.190:41411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:50:48.728112 13131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:50:48.728349 13131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:50:48.733578 13143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.733603 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:50:48.733603 13147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:50:48.733790 13131 server_base.cc:1034] running on GCE node
I20260812 14:50:48.734097 13131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:50:48.734342 13131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:50:48.735505 13131 hybrid_clock.cc:648] HybridClock initialized: now 1786546248735486 us; error 48 us; skew 500 ppm
Aug 12 14:50:48 dist-test-slave-8hhm krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:50:48.738155 13131 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:50:48.738596 13131 webserver.cc:460] Webserver started at http://127.13.146.131:39361/ using document root <none> and password file <none>
I20260812 14:50:48.738842 13131 fs_manager.cc:359] Metadata directory not provided
I20260812 14:50:48.738911 13131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:50:48.740278 13131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:48.740968 13157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.741168 13131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:50:48.741313 13131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "23d2851107984eeda2a52e24feec9e3c"
format_stamp: "Formatted at 2026-08-12 14:50:07 on dist-test-slave-8hhm"
server_key: "f11a29c31ba0678c8038a16f48ff76f6"
server_key_iv: "65e0222473c77e3481f9514ec00d31ba"
server_key_version: "encryptionkey@0"
I20260812 14:50:48.741684 13131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:50:48.756639 13131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:50:48.758684 13131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:50:48.758849 13131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:50:48.759121 13131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:50:48.759440 13131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:50:48.759492 13131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.759529 13131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:50:48.759581 13131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:50:48.765924 13131 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41815
I20260812 14:50:48.765977 13366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41815 every 8 connection(s)
I20260812 14:50:48.766345 13131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:50:48.767403 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.767010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:58801)
0812 14:50:48.767184 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.767260 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
W20260812 14:50:48.767838 13368 heartbeater.cc:644] Failed to heartbeat to 127.13.146.190:41411 (0 consecutive failures): Network error: Failed to ping master at 127.13.146.190:41411: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
I20260812 14:50:48.768118 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.768018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:53547)
0812 14:50:48.768071 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.768099 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:50:48.768356 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:48.768309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:57217)
0812 14:50:48.768328 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:48.768345 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 14:50:48.768451 13368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:50:48.775486 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 13131
I20260812 14:50:49.576764 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.576511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:54587)
0812 14:50:49.576651 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:49.576697 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260812 14:50:49.634485 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.634289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:58951)
0812 14:50:49.634401 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:49.634434 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260812 14:50:49.714821 13378 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:49.714978 13378 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:49.715456 13378 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
Aug 12 14:50:49 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.131@KRBTEST.COM
Aug 12 14:50:49 dist-test-slave-8hhm krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546248, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.130@KRBTEST.COM
I20260812 14:50:49.728475 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.715603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:41815 (local address 127.13.146.129:46165)
0812 14:50:49.715681 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:49.715694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:49.715754 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:49.716820 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:49.716823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:49.717408 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:49.717698 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:49.717706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.719409 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.719410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:49.720764 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:49.720768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.721625 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.721627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:49.721796 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:49.722461 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:49.722473 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:49.724439 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:49.726755 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:49.726763 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:49.726765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:49.727070 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:49.727383 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:49.727385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:49.727501 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:49.727921 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:49.727929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:49.728063 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 14:50:49.728172 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:50:49.728381 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":83}
I20260812 14:50:49.728605 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.715773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:38561 (local address 127.13.146.130:42689)
0812 14:50:49.715930 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:49.715938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:49.715970 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:50:49.716517 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:49.716520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:49.716580 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:49.716744 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:49.717843 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.719281 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.720862 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.721511 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.721627 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:49.724888 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:49.724894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:49.724895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:49.724909 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:49.726857 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727261 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:49.727265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:49.727265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:49.727368 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727584 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:49.727585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:49.727586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:49.727835 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:49.727926 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:49.728348 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 14:50:49.728494 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":51,"threads_started":1}
I20260812 14:50:49.728749 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.715644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:42689 (local address 127.13.146.129:38561)
0812 14:50:49.715786 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:49.715797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:50:49.715856 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:50:49.716820 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:50:49.716823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:50:49.717408 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:50:49.717698 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:49.717706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.719408 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.719409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:49.720764 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:50:49.720768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.721628 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.721629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:50:49.721796 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:49.722472 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:50:49.722477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:50:49.724742 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:50:49.726953 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:49.726955 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:49.726955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:49.727148 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:49.727450 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:50:49.727451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:50:49.727518 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:50:49.727919 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:50:49.727927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:50:49.728063 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 14:50:49.728178 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 14:50:49.728460 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":238,"thread_start_us":73,"threads_started":1}
I20260812 14:50:49.728935 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.715743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:46165 (local address 127.13.146.131:41815)
0812 14:50:49.715878 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 14:50:49.715890 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:50:49.715917 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:50:49.716511 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:50:49.716516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:50:49.716584 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:50:49.716744 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:50:49.717844 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.719281 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.720862 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:50:49.721511 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:50:49.721627 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:50:49.724575 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:50:49.724580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:50:49.724580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:50:49.724595 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:50:49.726644 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727170 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:49.727174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:49.727174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:49.727297 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:50:49.727577 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:50:49.727578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:50:49.727578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:50:49.727836 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:50:49.727927 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:50:49.728694 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 14:50:49.728837 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
W20260812 14:50:49.729885 12670 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
W20260812 14:50:49.730103 12673 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
I20260812 14:50:49.730161 12673 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e; no voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3
I20260812 14:50:49.730279 13378 raft_consensus.cc:2726] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:50:49.768970 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:49.768748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:51043)
0812 14:50:49.768886 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:49.768929 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260812 14:50:50.577561 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:50.577305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:52223)
0812 14:50:50.577471 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:50.577512 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260812 14:50:50.635251 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:50.634955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:35239)
0812 14:50:50.635121 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:50.635166 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260812 14:50:50.769721 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:50.769450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:44147)
0812 14:50:50.769598 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:50.769648 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260812 14:50:51.574004 13398 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:51.574123 13398 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:51.574366 13398 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
W20260812 14:50:51.574775 12673 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
W20260812 14:50:51.574862 12670 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
I20260812 14:50:51.574999 12670 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e; no voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3
I20260812 14:50:51.575091 13398 raft_consensus.cc:2726] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:50:51.578186 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:51.578002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:43109)
0812 14:50:51.578115 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:51.578150 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260812 14:50:51.636047 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:51.635773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:35281)
0812 14:50:51.635936 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:51.635989 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260812 14:50:51.770418 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:51.770202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:43985)
0812 14:50:51.770338 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:51.770376 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260812 14:50:52.578943 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:52.578609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:58799)
0812 14:50:52.578822 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:52.578871 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
I20260812 14:50:52.636871 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:52.636652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:39707)
0812 14:50:52.636786 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:52.636828 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260812 14:50:52.771318 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:52.771094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:48957)
0812 14:50:52.771235 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:52.771276 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260812 14:50:53.579768 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:53.579528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:48305)
0812 14:50:53.579686 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:53.579721 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260812 14:50:53.637660 13421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:53.637404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:44787)
0812 14:50:53.637546 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:53.637597 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260812 14:50:53.772038 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:53.771778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:56833)
0812 14:50:53.771947 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:53.771990 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260812 14:50:53.777720 13423 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:53.777817 13423 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:53.777980 13423 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
W20260812 14:50:53.778386 12670 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
W20260812 14:50:53.778506 12673 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
I20260812 14:50:53.778581 12673 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e; no voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3
I20260812 14:50:53.778674 13423 raft_consensus.cc:2726] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:50:54.580626 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:54.580357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:59895)
0812 14:50:54.580525 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:54.580577 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260812 14:50:54.638396 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:54.638142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:55191)
0812 14:50:54.638307 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:54.638348 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260812 14:50:54.772841 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:54.772611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:52089)
0812 14:50:54.772758 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:54.772793 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260812 14:50:55.581470 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:55.581153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:47249)
0812 14:50:55.581382 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:55.581421 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":163,"threads_started":1}
I20260812 14:50:55.639101 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:55.638875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:40629)
0812 14:50:55.639006 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:55.639054 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260812 14:50:55.773602 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:55.773369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:52801)
0812 14:50:55.773517 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:55.773556 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260812 14:50:55.924250 13436 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:50:55.924360 13436 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:50:55.924549 13436 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
W20260812 14:50:55.925001 12673 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
W20260812 14:50:55.925086 12670 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
I20260812 14:50:55.925134 12670 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e; no voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3
I20260812 14:50:55.925251 13436 raft_consensus.cc:2726] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:50:56.582335 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:56.582082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:57101)
0812 14:50:56.582243 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:56.582285 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260812 14:50:56.639909 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:56.639682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:45669)
0812 14:50:56.639818 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:56.639859 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260812 14:50:56.774281 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:56.774054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:59619)
0812 14:50:56.774201 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:56.774233 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260812 14:50:57.583304 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:57.583044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:53503)
0812 14:50:57.583201 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:57.583249 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260812 14:50:57.640586 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:57.640380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:50045)
0812 14:50:57.640504 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:57.640541 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260812 14:50:57.775027 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:57.774795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:55895)
0812 14:50:57.774939 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:57.774980 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 14:50:58.584115 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:58.583869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:52623)
0812 14:50:58.584019 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:58.584064 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260812 14:50:58.641290 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:58.641024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:60497)
0812 14:50:58.641153 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:58.641194 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260812 14:50:58.775919 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:58.775646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:41343)
0812 14:50:58.775820 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:58.775869 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260812 14:50:59.584926 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:59.584695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:40061)
0812 14:50:59.584833 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:59.584876 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260812 14:50:59.641971 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:59.641723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:51835)
0812 14:50:59.641879 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:59.641921 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260812 14:50:59.776721 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:50:59.776469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:54123)
0812 14:50:59.776630 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:50:59.776672 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260812 14:51:00.585776 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:00.585516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:43869)
0812 14:51:00.585677 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:00.585726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260812 14:51:00.642768 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:00.642518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:39339)
0812 14:51:00.642668 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:00.642714 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260812 14:51:00.777514 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:00.777199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:35773)
0812 14:51:00.777422 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:00.777461 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":150,"threads_started":1}
I20260812 14:51:01.587623 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:01.586222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:45579)
0812 14:51:01.587528 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:01.587576 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1290,"thread_start_us":1224,"threads_started":1}
I20260812 14:51:01.643604 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:01.643344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:59997)
0812 14:51:01.643510 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:01.643552 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
I20260812 14:51:01.778421 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:01.778110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:33419)
0812 14:51:01.778278 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:01.778327 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260812 14:51:02.588384 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:02.588126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:43477)
0812 14:51:02.588295 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:02.588334 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260812 14:51:02.644526 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:02.644278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:53267)
0812 14:51:02.644432 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:02.644475 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260812 14:51:02.779147 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:02.778913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:59955)
0812 14:51:02.779057 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:02.779096 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260812 14:51:02.939798 13473 raft_consensus.cc:482] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:02.939906 13473 raft_consensus.cc:504] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d2851107984eeda2a52e24feec9e3c" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41815 } } peers { permanent_uuid: "2492c2247bda4c999231969d2a2d13d3" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 42689 } } peers { permanent_uuid: "07e6ea22be8b490d91bcb2153d06cd7e" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43673 } }
I20260812 14:51:02.940088 13473 leader_election.cc:288] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815), 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689)
W20260812 14:51:02.940510 12670 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23d2851107984eeda2a52e24feec9e3c (127.13.146.131:41815): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
W20260812 14:51:02.940590 12673 leader_election.cc:341] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2492c2247bda4c999231969d2a2d13d3 (127.13.146.130:42689): Not found: Tablet not found: 2543ec6ca246462a864b04b80c00b322
I20260812 14:51:02.940696 12673 leader_election.cc:302] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e6ea22be8b490d91bcb2153d06cd7e; no voters: 23d2851107984eeda2a52e24feec9e3c, 2492c2247bda4c999231969d2a2d13d3
I20260812 14:51:02.940763 13473 raft_consensus.cc:2726] T 2543ec6ca246462a864b04b80c00b322 P 07e6ea22be8b490d91bcb2153d06cd7e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:51:03.589404 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:03.588987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.129:55389)
0812 14:51:03.589141 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:03.589195 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260812 14:51:03.645283 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:03.645047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.130:55749)
0812 14:51:03.645181 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:03.645236 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260812 14:51:03.779966 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:03.779734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:41411 (local address 127.13.146.131:36409)
0812 14:51:03.779881 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:03.779920 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.190:41411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260812 14:51:03.785915 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 12570
I20260812 14:51:03.802367 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 12885
I20260812 14:51:03.817934 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 13131
2026-08-12T14:51:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58867 ms)
[----------] 30 tests from SecurityITest (243321 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:03 dist-test-slave-8hhm krb5kdc[13480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:03 dist-test-slave-8hhm krb5kdc[13480](info): set up 2 sockets
Aug 12 14:51:03 dist-test-slave-8hhm krb5kdc[13480](info): commencing operation
krb5kdc: starting...
W20260812 14:51:05.158815 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:05.290429 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:40027
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:40027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:05.375989 13498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:05.376216 13498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:05.376242 13498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:05.377490 13498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:05.377539 13498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:05.377553 13498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:05.377566 13498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:05.377578 13498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:05.379185 13498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:40027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:40027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:05.379446 13498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:05.379724 13498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:05.385064 13509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.385064 13510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.385165 13516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:05.385232 13498 server_base.cc:1034] running on GCE node
I20260812 14:51:05.385627 13498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:05.385957 13498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:05.387122 13498 hybrid_clock.cc:648] HybridClock initialized: now 1786546265387112 us; error 32 us; skew 500 ppm
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:05.389865 13498 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:05.390376 13498 webserver.cc:460] Webserver started at http://127.13.146.190:33365/ using document root <none> and password file <none>
I20260812 14:51:05.390672 13498 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:05.390745 13498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:05.390897 13498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:05.392025 13498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "66625d671fb14f17ad9fce832a33452a"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1cdf5501264215b5579bf07ae1ea2040"
server_key_iv: "87c9c42e52cc7cfb2514fcc21cab7471"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.392421 13498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "66625d671fb14f17ad9fce832a33452a"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1cdf5501264215b5579bf07ae1ea2040"
server_key_iv: "87c9c42e52cc7cfb2514fcc21cab7471"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.393883 13498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:05.394693 13530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.394888 13498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:05.395000 13498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "66625d671fb14f17ad9fce832a33452a"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1cdf5501264215b5579bf07ae1ea2040"
server_key_iv: "87c9c42e52cc7cfb2514fcc21cab7471"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.395077 13498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:05.409808 13498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:05.411911 13498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:05.412086 13498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:05.416594 13498 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:40027
I20260812 14:51:05.416657 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:40027 every 8 connection(s)
I20260812 14:51:05.417055 13498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:05.417997 13618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:05.420429 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: Bootstrap starting.
I20260812 14:51:05.421044 13618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:05.421406 13618 log.cc:824] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:05.422216 13618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: No bootstrap required, opened a new log
I20260812 14:51:05.423905 13618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } }
I20260812 14:51:05.424063 13618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:05.424091 13618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66625d671fb14f17ad9fce832a33452a, State: Initialized, Role: FOLLOWER
I20260812 14:51:05.424207 13618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } }
I20260812 14:51:05.424309 13618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:05.424348 13618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:05.424412 13618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:05.425045 13618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } }
I20260812 14:51:05.425176 13618 leader_election.cc:302] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66625d671fb14f17ad9fce832a33452a; no voters:
I20260812 14:51:05.425417 13618 leader_election.cc:288] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:05.425465 13625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:05.425643 13625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [term 1 LEADER]: Becoming Leader. State: Replica: 66625d671fb14f17ad9fce832a33452a, State: Running, Role: LEADER
I20260812 14:51:05.425724 13618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:05.425817 13625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } }
I20260812 14:51:05.426290 13629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66625d671fb14f17ad9fce832a33452a. Latest consensus state: current_term: 1 leader_uuid: "66625d671fb14f17ad9fce832a33452a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } } }
I20260812 14:51:05.426386 13629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:05.426426 13628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66625d671fb14f17ad9fce832a33452a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66625d671fb14f17ad9fce832a33452a" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 40027 } } }
I20260812 14:51:05.426496 13628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:05.426659 13640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:05.426728 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 13498
I20260812 14:51:05.426843 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:05.427083 13898 external_mini_cluster.cc:1428] Setting key 36f57f2b0c683f9f7db1da50cbc00a6a
I20260812 14:51:05.427248 13640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:05.432638 13640 catalog_manager.cc:1347] Generated new cluster ID: a035f9abecb942ec91f3d53111b77b66
I20260812 14:51:05.432726 13640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:05.438876 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.427601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.13.146.190:40027)
0812 14:51:05.427790 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:05.427801 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:05.428127 (+ 326us) server_negotiation.cc:407] Connection header received
0812 14:51:05.428248 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:05.428255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:05.428312 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:05.428456 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:05.429295 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.430191 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.431319 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.431791 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.431919 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.434848 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:05.434854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:05.434855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:05.434872 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:05.437051 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.437487 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.437491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.437491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.437631 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.437862 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.437864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.437864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.438206 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:05.438405 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:05.438479 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:51:05.438628 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260812 14:51:05.439280 13640 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:05.439992 13640 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:05.450397 13640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: Generated new TSK 0
I20260812 14:51:05.450649 13640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:51:05.486397 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:05.581943 13664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:05.582186 13664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:05.582228 13664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:05.583406 13664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:05.583487 13664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:05.583532 13664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:05.585067 13664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:05.585387 13664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:05.585640 13664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:05.590952 13673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:05.591034 13664 server_base.cc:1034] running on GCE node
W20260812 14:51:05.590983 13676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.590983 13674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:05.591418 13664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:05.591651 13664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:05.592839 13664 hybrid_clock.cc:648] HybridClock initialized: now 1786546265592813 us; error 41 us; skew 500 ppm
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:05.599612 13664 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:51:05.600127 13664 webserver.cc:460] Webserver started at http://127.13.146.129:45125/ using document root <none> and password file <none>
I20260812 14:51:05.600349 13664 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:05.600391 13664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:05.600481 13664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:05.601648 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "3d48770ee2a140a7b3571519808c30a8"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "5d2c261b50ae9288274ea2f9dfd6788f"
server_key_iv: "703e68bd021ce35af5651c131e3d3f56"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.601963 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "3d48770ee2a140a7b3571519808c30a8"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "5d2c261b50ae9288274ea2f9dfd6788f"
server_key_iv: "703e68bd021ce35af5651c131e3d3f56"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.603269 13664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:05.603972 13686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.604172 13664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:05.604250 13664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "3d48770ee2a140a7b3571519808c30a8"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "5d2c261b50ae9288274ea2f9dfd6788f"
server_key_iv: "703e68bd021ce35af5651c131e3d3f56"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.604347 13664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:05.626088 13664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:05.628795 13664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:05.628983 13664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:05.629280 13664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:05.629643 13664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:05.629683 13664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.629736 13664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:05.629760 13664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.635994 13664 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:34041
I20260812 14:51:05.636086 13907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:34041 every 8 connection(s)
I20260812 14:51:05.636457 13664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:05.643817 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 13664
I20260812 14:51:05.643950 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:05.644214 13898 external_mini_cluster.cc:1428] Setting key 77060c317a84b8a20d6488d3f5fc52a5
I20260812 14:51:05.647576 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.637334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:60795 (local address 127.13.146.190:40027)
0812 14:51:05.637385 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:05.637388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:05.637737 (+ 349us) server_negotiation.cc:407] Connection header received
0812 14:51:05.638253 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:05.638255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:05.638271 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:05.638332 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:05.639372 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.639889 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.641316 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.641540 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.641610 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.644130 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:05.644133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:05.644133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:05.644143 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:05.645892 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.646382 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.646384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.646384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.646456 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.646742 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.646743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.646744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.646923 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:05.646997 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:05.647419 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 14:51:05.647505 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:51:05.647887 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.637338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:40027 (local address 127.13.146.129:60795)
0812 14:51:05.637567 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:05.637614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:05.638141 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:05.638391 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:05.638395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:05.639039 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:05.639250 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.639256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.640006 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.640007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.641150 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.641153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.641670 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.641671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.641803 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.642445 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:05.642453 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:05.644011 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:05.645988 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.645998 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.646001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.646278 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.646534 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.646535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.646535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.646661 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.647031 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:05.647041 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:05.647242 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 14:51:05.647373 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 14:51:05.647581 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260812 14:51:05.648424 13908 heartbeater.cc:347] Connected to a master server at 127.13.146.190:40027
I20260812 14:51:05.648527 13908 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:05.648742 13908 heartbeater.cc:511] Master 127.13.146.190:40027 requested a full tablet report, sending...
I20260812 14:51:05.649451 13566 ts_manager.cc:194] Registered new tserver with Master: 3d48770ee2a140a7b3571519808c30a8 (127.13.146.129:34041)
I20260812 14:51:05.650101 13566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:60795
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:51:05.676641 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:05.762173 13915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:05.762395 13915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:05.762420 13915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:05.763558 13915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:05.763597 13915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:05.763624 13915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:05.765053 13915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:05.765372 13915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:05.765579 13915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:05.770915 13921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.770907 13922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.770910 13924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:05.771119 13915 server_base.cc:1034] running on GCE node
I20260812 14:51:05.771441 13915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:05.771720 13915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:05.772902 13915 hybrid_clock.cc:648] HybridClock initialized: now 1786546265772878 us; error 40 us; skew 500 ppm
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:05.775682 13915 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:51:05.776208 13915 webserver.cc:460] Webserver started at http://127.13.146.130:41807/ using document root <none> and password file <none>
I20260812 14:51:05.776467 13915 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:05.776537 13915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:05.776693 13915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:05.777859 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "1791cee1b1b24087ab5dd46f7571e783"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "69eff341b13bc419f3b5dfe3d13ad404"
server_key_iv: "841e8efa66ac69d235d9bf5ccb2f3bd5"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.778244 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "1791cee1b1b24087ab5dd46f7571e783"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "69eff341b13bc419f3b5dfe3d13ad404"
server_key_iv: "841e8efa66ac69d235d9bf5ccb2f3bd5"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.779652 13915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:51:05.780478 13936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.780678 13915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:05.780792 13915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "1791cee1b1b24087ab5dd46f7571e783"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "69eff341b13bc419f3b5dfe3d13ad404"
server_key_iv: "841e8efa66ac69d235d9bf5ccb2f3bd5"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.780872 13915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:05.793728 13915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:05.795984 13915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:05.796161 13915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:05.796422 13915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:05.796757 13915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:05.796814 13915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.796861 13915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:05.796885 13915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.803328 13915 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:40157
I20260812 14:51:05.803375 14126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:40157 every 8 connection(s)
I20260812 14:51:05.803735 13915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:05.813009 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 13915
I20260812 14:51:05.813160 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:05.813468 13898 external_mini_cluster.cc:1428] Setting key 43c5d96b9b11ee33d99ff5c9fb10fe2e
I20260812 14:51:05.814036 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.804541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:57619 (local address 127.13.146.190:40027)
0812 14:51:05.804613 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:05.804615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:05.804808 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:51:05.805320 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:05.805321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:05.805332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:05.805402 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:05.806355 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.806879 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.807965 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.808205 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.808272 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.810543 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:05.810545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:05.810545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:05.810553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:05.812137 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.812525 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.812526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.812527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.812604 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.813273 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.813275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.813275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.813458 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:05.813551 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:05.813867 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 14:51:05.813969 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:51:05.814289 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.804501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:40027 (local address 127.13.146.130:57619)
0812 14:51:05.804669 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:05.804714 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:05.805234 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:05.805458 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:05.805461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:05.806062 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:05.806261 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.806267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.807026 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.807027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.807856 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.807858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.808289 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.808290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.808406 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.808914 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:05.808922 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:05.810421 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:05.812211 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.812217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.812219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.812436 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.812683 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.812683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.812684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.812802 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.813560 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:05.813567 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:05.813727 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 14:51:05.813823 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 14:51:05.814024 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260812 14:51:05.814947 14128 heartbeater.cc:347] Connected to a master server at 127.13.146.190:40027
I20260812 14:51:05.815078 14128 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:05.815308 14128 heartbeater.cc:511] Master 127.13.146.190:40027 requested a full tablet report, sending...
I20260812 14:51:05.815740 13566 ts_manager.cc:194] Registered new tserver with Master: 1791cee1b1b24087ab5dd46f7571e783 (127.13.146.130:40157)
I20260812 14:51:05.816133 13566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:57619
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:51:05.844233 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:05.929558 14136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:05.929827 14136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:05.929872 14136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:05.931098 14136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:05.931180 14136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:05.931227 14136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:05.932854 14136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:40027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:05.933200 14136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:05.933472 14136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:51:05.938985 14136 server_base.cc:1034] running on GCE node
W20260812 14:51:05.938934 14149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.939168 14154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:05.938930 14150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:05.939482 14136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:05.939760 14136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:05.940964 14136 hybrid_clock.cc:648] HybridClock initialized: now 1786546265940944 us; error 27 us; skew 500 ppm
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:05.943984 14136 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:51:05.944545 14136 webserver.cc:460] Webserver started at http://127.13.146.131:45019/ using document root <none> and password file <none>
I20260812 14:51:05.944808 14136 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:05.944887 14136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:05.945060 14136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:05.946347 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "625a78ac73944f039397ca9d43c68898"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1774e7e885594e205db78142680570a7"
server_key_iv: "86498586e85eeb6767b8f6941d3e7957"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.946774 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "625a78ac73944f039397ca9d43c68898"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1774e7e885594e205db78142680570a7"
server_key_iv: "86498586e85eeb6767b8f6941d3e7957"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.948287 14136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:51:05.949074 14169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.949306 14136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:05.949414 14136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "625a78ac73944f039397ca9d43c68898"
format_stamp: "Formatted at 2026-08-12 14:51:05 on dist-test-slave-8hhm"
server_key: "1774e7e885594e205db78142680570a7"
server_key_iv: "86498586e85eeb6767b8f6941d3e7957"
server_key_version: "encryptionkey@0"
I20260812 14:51:05.949502 14136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:05.967785 14136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:05.970414 14136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:05.970587 14136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:05.970847 14136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:05.971200 14136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:05.971257 14136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.971302 14136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:05.971326 14136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:05.977595 14136 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:42167
I20260812 14:51:05.977639 14371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:42167 every 8 connection(s)
I20260812 14:51:05.977988 14136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:51:05.980473 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14136
I20260812 14:51:05.980613 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:05.980839 13898 external_mini_cluster.cc:1428] Setting key 3d5ecdc2af73640a779dab68422f5a8d
Aug 12 14:51:05 dist-test-slave-8hhm krb5kdc[13480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546265, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:05.988494 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.978777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:56059 (local address 127.13.146.190:40027)
0812 14:51:05.978852 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:05.978854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:05.979088 (+ 234us) server_negotiation.cc:407] Connection header received
0812 14:51:05.979581 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:05.979582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:05.979591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:05.979654 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:05.980851 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.981687 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.982795 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.982980 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.983038 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.985517 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:05.985518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:05.985519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:05.985526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:05.987185 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.987601 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.987602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.987602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.987664 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.987921 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.987922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.987922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.988053 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:05.988112 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:05.988355 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 14:51:05.988419 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:51:05.988729 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.978727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:40027 (local address 127.13.146.131:56059)
0812 14:51:05.978901 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:05.978945 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:05.979491 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:05.979710 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:05.979713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:05.980406 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:05.980724 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.980745 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.981782 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.981783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.982712 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:05.982715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.983061 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.983062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:05.983165 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.983760 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:05.983768 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:05.985390 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:05.987268 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.987275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.987276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.987503 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.987738 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:05.987739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:05.987739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:05.987849 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:05.988115 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:05.988120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:05.988256 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 14:51:05.988332 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:51:05.988459 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260812 14:51:05.989336 14372 heartbeater.cc:347] Connected to a master server at 127.13.146.190:40027
I20260812 14:51:05.989450 14372 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:05.989637 14372 heartbeater.cc:511] Master 127.13.146.190:40027 requested a full tablet report, sending...
I20260812 14:51:05.990074 13566 ts_manager.cc:194] Registered new tserver with Master: 625a78ac73944f039397ca9d43c68898 (127.13.146.131:42167)
I20260812 14:51:05.990492 13566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:56059
I20260812 14:51:05.992836 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:06.000715 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:05.994398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.13.146.190:40027)
0812 14:51:05.994455 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:05.994457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:05.994579 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:51:05.994632 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:05.994633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:05.994642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:05.994692 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:05.995246 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.995701 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.996484 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:05.996657 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:05.996705 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:05.998207 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:05.998208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:05.998208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:05.998214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:05.999580 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:05.999945 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:05.999946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:05.999946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:05.999997 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:06.000204 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:06.000205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:06.000205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:06.000343 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.000420 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.000543 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.000631 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:51:06.002556 13566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:06.004204 13566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:06.011051 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.006889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:42167 (local address 127.0.0.1:57964)
0812 14:51:06.007229 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.007349 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.007422 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.007631 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.007632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.007634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.007675 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.007676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.008989 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.008990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.010059 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.010061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.010738 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.010819 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.010834 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.010906 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.010971 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260812 14:51:06.011067 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.007047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.13.146.131:42167)
0812 14:51:06.007279 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.007284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.007422 (+ 138us) server_negotiation.cc:407] Connection header received
0812 14:51:06.007495 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.007498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.007544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.007620 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.007733 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.008878 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010147 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.010626 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010737 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.010800 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.010927 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.010976 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260812 14:51:06.011144 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.006882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.13.146.129:34041)
0812 14:51:06.007053 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.007059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.007183 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:51:06.007291 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.007294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.007311 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.007365 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.007671 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.008529 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.009964 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.010695 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010771 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.010823 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.011011 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.011057 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260812 14:51:06.011416 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.006778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.13.146.130:40157)
0812 14:51:06.007611 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.007615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.007624 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:06.007663 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.007665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.007682 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.007750 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.007959 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.009004 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010175 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.010869 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010932 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.010976 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.011325 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.011359 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":113,"threads_started":1}
I20260812 14:51:06.011263 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.006673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:40157 (local address 127.0.0.1:47116)
0812 14:51:06.006856 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.007007 (+ 151us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.007137 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.007833 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.007834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.007836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.007876 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.007877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.009109 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.009110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.010075 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.010078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.011012 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.011013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.011092 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.011096 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.011150 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.011190 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":8,"thread_start_us":80,"threads_started":1}
I20260812 14:51:06.011369 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.006674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:34041 (local address 127.0.0.1:50828)
0812 14:51:06.007082 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.007092 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.007183 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.007496 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.007499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.007501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.007567 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.007569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.008630 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.008631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.009866 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.009869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.010850 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.010850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.010912 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.010915 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.011260 (+ 345us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.011317 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":65,"threads_started":1}
I20260812 14:51:06.012327 14018 tablet_service.cc:1467] Processing CreateTablet for tablet e1619d6df11141cf9cbee34a7a262739 (DEFAULT_TABLE table=test-table [id=69be5e3221204947872776503a1febd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.012477 13776 tablet_service.cc:1467] Processing CreateTablet for tablet e1619d6df11141cf9cbee34a7a262739 (DEFAULT_TABLE table=test-table [id=69be5e3221204947872776503a1febd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.012732 14018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1619d6df11141cf9cbee34a7a262739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.012881 13776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1619d6df11141cf9cbee34a7a262739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.013684 14255 tablet_service.cc:1467] Processing CreateTablet for tablet e1619d6df11141cf9cbee34a7a262739 (DEFAULT_TABLE table=test-table [id=69be5e3221204947872776503a1febd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.014035 14255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1619d6df11141cf9cbee34a7a262739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.015702 14400 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Bootstrap starting.
I20260812 14:51:06.016366 14402 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Bootstrap starting.
I20260812 14:51:06.016434 14400 tablet_bootstrap.cc:654] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.017088 14400 log.cc:824] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.017184 14402 tablet_bootstrap.cc:654] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.017187 14408 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Bootstrap starting.
I20260812 14:51:06.017879 14402 log.cc:824] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.017908 14408 tablet_bootstrap.cc:654] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.018193 14400 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: No bootstrap required, opened a new log
I20260812 14:51:06.018287 14400 ts_tablet_manager.cc:1397] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:51:06.018440 14408 log.cc:824] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.018807 14402 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: No bootstrap required, opened a new log
I20260812 14:51:06.018890 14402 ts_tablet_manager.cc:1397] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:51:06.019236 14408 tablet_bootstrap.cc:492] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: No bootstrap required, opened a new log
I20260812 14:51:06.019316 14408 ts_tablet_manager.cc:1397] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:51:06.020349 14402 raft_consensus.cc:348] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.020533 14402 raft_consensus.cc:374] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.020427 14400 raft_consensus.cc:348] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.020594 14400 raft_consensus.cc:374] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.020603 14402 raft_consensus.cc:729] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d48770ee2a140a7b3571519808c30a8, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.020685 14400 raft_consensus.cc:729] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1791cee1b1b24087ab5dd46f7571e783, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.020725 14402 consensus_queue.cc:260] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.020879 14400 consensus_queue.cc:260] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.021037 13908 heartbeater.cc:503] Master 127.13.146.190:40027 was elected leader, sending a full tablet report...
I20260812 14:51:06.021286 14400 ts_tablet_manager.cc:1428] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:51:06.021325 14128 heartbeater.cc:503] Master 127.13.146.190:40027 was elected leader, sending a full tablet report...
I20260812 14:51:06.021430 14408 raft_consensus.cc:348] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.021600 14408 raft_consensus.cc:374] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.021673 14402 ts_tablet_manager.cc:1428] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:51:06.021705 14408 raft_consensus.cc:729] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625a78ac73944f039397ca9d43c68898, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.021844 14408 consensus_queue.cc:260] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.022190 14372 heartbeater.cc:503] Master 127.13.146.190:40027 was elected leader, sending a full tablet report...
I20260812 14:51:06.022179 14408 ts_tablet_manager.cc:1428] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260812 14:51:06.054296 14129 tablet.cc:2369] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:51:06.137331 13909 tablet.cc:2369] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:06.215663 14420 raft_consensus.cc:482] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:06.215826 14420 raft_consensus.cc:504] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.216233 14420 leader_election.cc:288] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d48770ee2a140a7b3571519808c30a8 (127.13.146.129:34041), 625a78ac73944f039397ca9d43c68898 (127.13.146.131:42167)
I20260812 14:51:06.220645 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.216701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:33051 (local address 127.13.146.131:42167)
0812 14:51:06.216776 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.216778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.216904 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:51:06.216943 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.216944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.216962 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.217059 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.217171 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.217961 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.219562 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.220218 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.220311 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.220322 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.220514 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.220562 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:51:06.220645 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.216577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:38095 (local address 127.13.146.129:34041)
0812 14:51:06.216672 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.216674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.216687 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:51:06.216751 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.216752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.216766 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.216866 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.217046 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.217928 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.219562 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.220208 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.220311 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.220322 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.220522 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.220567 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 14:51:06.220624 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.216444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:34041 (local address 127.13.146.130:38095)
0812 14:51:06.216511 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.216524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.216574 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.216877 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.216878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.216881 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.216961 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.216965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.218037 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.218038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.219425 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.219428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.220315 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.220316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.220429 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.220432 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.220486 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.220523 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":141}
I20260812 14:51:06.220630 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.216583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:42167 (local address 127.13.146.130:33051)
0812 14:51:06.216806 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.216814 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.216894 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.217071 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.217072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.217073 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.217120 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.217122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.218056 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.218056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.219425 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.219428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.220315 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.220316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.220411 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.220415 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.220477 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.220523 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":46,"thread_start_us":137,"threads_started":1}
I20260812 14:51:06.221095 14291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1619d6df11141cf9cbee34a7a262739" candidate_uuid: "1791cee1b1b24087ab5dd46f7571e783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625a78ac73944f039397ca9d43c68898" is_pre_election: true
I20260812 14:51:06.221095 13805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1619d6df11141cf9cbee34a7a262739" candidate_uuid: "1791cee1b1b24087ab5dd46f7571e783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d48770ee2a140a7b3571519808c30a8" is_pre_election: true
I20260812 14:51:06.221298 14291 raft_consensus.cc:2454] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1791cee1b1b24087ab5dd46f7571e783 in term 0.
I20260812 14:51:06.221298 13805 raft_consensus.cc:2454] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1791cee1b1b24087ab5dd46f7571e783 in term 0.
I20260812 14:51:06.221575 13942 leader_election.cc:302] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1791cee1b1b24087ab5dd46f7571e783, 625a78ac73944f039397ca9d43c68898; no voters:
I20260812 14:51:06.221710 14420 raft_consensus.cc:2781] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:06.221769 14420 raft_consensus.cc:482] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:06.221797 14420 raft_consensus.cc:3037] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:06.222566 14420 raft_consensus.cc:504] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.222718 14420 leader_election.cc:288] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [CANDIDATE]: Term 1 election: Requested vote from peers 3d48770ee2a140a7b3571519808c30a8 (127.13.146.129:34041), 625a78ac73944f039397ca9d43c68898 (127.13.146.131:42167)
I20260812 14:51:06.222941 14291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1619d6df11141cf9cbee34a7a262739" candidate_uuid: "1791cee1b1b24087ab5dd46f7571e783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625a78ac73944f039397ca9d43c68898"
I20260812 14:51:06.222941 13805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1619d6df11141cf9cbee34a7a262739" candidate_uuid: "1791cee1b1b24087ab5dd46f7571e783" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d48770ee2a140a7b3571519808c30a8"
I20260812 14:51:06.223028 14291 raft_consensus.cc:3037] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:06.223028 13805 raft_consensus.cc:3037] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:06.223960 14291 raft_consensus.cc:2454] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1791cee1b1b24087ab5dd46f7571e783 in term 1.
I20260812 14:51:06.223960 13805 raft_consensus.cc:2454] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1791cee1b1b24087ab5dd46f7571e783 in term 1.
I20260812 14:51:06.224177 13942 leader_election.cc:302] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1791cee1b1b24087ab5dd46f7571e783, 625a78ac73944f039397ca9d43c68898; no voters:
I20260812 14:51:06.224287 14420 raft_consensus.cc:2781] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:06.224447 14420 raft_consensus.cc:686] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 1 LEADER]: Becoming Leader. State: Replica: 1791cee1b1b24087ab5dd46f7571e783, State: Running, Role: LEADER
I20260812 14:51:06.224555 14420 consensus_queue.cc:237] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } }
I20260812 14:51:06.225550 13566 catalog_manager.cc:5495] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1791cee1b1b24087ab5dd46f7571e783 (127.13.146.130). New cstate: current_term: 1 leader_uuid: "1791cee1b1b24087ab5dd46f7571e783" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1791cee1b1b24087ab5dd46f7571e783" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 40157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 } health_report { overall_health: UNKNOWN } } }
W20260812 14:51:06.228533 14374 tablet.cc:2369] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:06.237733 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.235512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.13.146.130:40157)
0812 14:51:06.235570 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.235574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.235586 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:51:06.235618 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.235619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.235636 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.235696 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:06.235813 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.236273 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.237015 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.237197 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.237301 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.237424 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:06.237528 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:06.237591 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.237622 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.237639 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:51:06.239605 14291 raft_consensus.cc:1261] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 1 FOLLOWER]: Refusing update from remote peer 1791cee1b1b24087ab5dd46f7571e783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:06.239625 13805 raft_consensus.cc:1261] T e1619d6df11141cf9cbee34a7a262739 P 3d48770ee2a140a7b3571519808c30a8 [term 1 FOLLOWER]: Refusing update from remote peer 1791cee1b1b24087ab5dd46f7571e783: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:06.240017 14420 consensus_queue.cc:1035] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "625a78ac73944f039397ca9d43c68898" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:06.240168 14426 consensus_queue.cc:1035] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d48770ee2a140a7b3571519808c30a8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 34041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:06.244097 14433 mvcc.cc:205] Tried to move back new op lower bound from 7317693506514415616 to 7317693506456576000. Current Snapshot: MvccSnapshot[applied={T|T < 7317693506514415616 or (T in {7317693506514415616})}]
I20260812 14:51:06.245749 13565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:06.245862 13565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:06.246687 13565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66625d671fb14f17ad9fce832a33452a: Sending DeleteTablet for 3 replicas of tablet e1619d6df11141cf9cbee34a7a262739
I20260812 14:51:06.247109 13776 tablet_service.cc:1514] Processing DeleteTablet for tablet e1619d6df11141cf9cbee34a7a262739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:06 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:50828
I20260812 14:51:06.247102 14018 tablet_service.cc:1514] Processing DeleteTablet for tablet e1619d6df11141cf9cbee34a7a262739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:06 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:47116
I20260812 14:51:06.247342 14442 tablet_replica.cc:330] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: stopping tablet replica
I20260812 14:51:06.247349 14255 tablet_service.cc:1514] Processing DeleteTablet for tablet e1619d6df11141cf9cbee34a7a262739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:06 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57964
I20260812 14:51:06.247450 14442 raft_consensus.cc:2229] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:51:06.247515 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 13664
I20260812 14:51:06.247553 14442 raft_consensus.cc:2258] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:06.259388 14442 ts_tablet_manager.cc:1905] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:06.264971 14442 ts_tablet_manager.cc:1918] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:06.265090 14442 log.cc:1196] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/e1619d6df11141cf9cbee34a7a262739
I20260812 14:51:06.265439 14442 ts_tablet_manager.cc:1939] T e1619d6df11141cf9cbee34a7a262739 P 1791cee1b1b24087ab5dd46f7571e783: Deleting consensus metadata
I20260812 14:51:06.265995 13536 catalog_manager.cc:4841] TS 1791cee1b1b24087ab5dd46f7571e783 (127.13.146.130:40157): tablet e1619d6df11141cf9cbee34a7a262739 (table test-table [id=69be5e3221204947872776503a1febd3]) successfully deleted
I20260812 14:51:06.269696 14445 tablet_replica.cc:330] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: stopping tablet replica
I20260812 14:51:06.269884 14445 raft_consensus.cc:2229] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:06.270051 14445 raft_consensus.cc:2258] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:06.270552 14445 ts_tablet_manager.cc:1905] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 14:51:06.271045 13537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:34041 (error 108)
I20260812 14:51:06.271384 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 13915
I20260812 14:51:06.278203 14445 ts_tablet_manager.cc:1918] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:06.278326 14445 log.cc:1196] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/e1619d6df11141cf9cbee34a7a262739
I20260812 14:51:06.278429 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.278209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:34041 (local address 127.0.0.1:50840)
0812 14:51:06.278305 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.278372 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:34041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 14:51:06.278609 13537 catalog_manager.cc:4584] TS 3d48770ee2a140a7b3571519808c30a8 (127.13.146.129:34041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1619d6df11141cf9cbee34a7a262739: Network error: Client connection negotiation failed: client connection to 127.13.146.129:34041: connect: Connection refused (error 111)
I20260812 14:51:06.278697 14445 ts_tablet_manager.cc:1939] T e1619d6df11141cf9cbee34a7a262739 P 625a78ac73944f039397ca9d43c68898: Deleting consensus metadata
I20260812 14:51:06.279229 13537 catalog_manager.cc:4841] TS 625a78ac73944f039397ca9d43c68898 (127.13.146.131:42167): tablet e1619d6df11141cf9cbee34a7a262739 (table test-table [id=69be5e3221204947872776503a1febd3]) successfully deleted
I20260812 14:51:06.283207 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14136
I20260812 14:51:06.313104 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 13498
2026-08-12T14:51:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2492 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T14:51:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:06Z Disabled control of system clock
I20260812 14:51:06.353467 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46467
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:06.438459 14456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:06.438761 14456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:06.438798 14456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:06.439975 14456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:06.440054 14456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:06.440076 14456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:06.440099 14456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:06.440127 14456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:06.441550 14456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:06.441848 14456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:06.442163 14456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:51:06.447537 14456 server_base.cc:1034] running on GCE node
W20260812 14:51:06.447495 14464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.447503 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.447495 14463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:06.447999 14456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:06.448346 14456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:06.449496 14456 hybrid_clock.cc:648] HybridClock initialized: now 1786546266449468 us; error 37 us; skew 500 ppm
I20260812 14:51:06.451126 14456 webserver.cc:460] Webserver started at http://127.13.146.190:46783/ using document root <none> and password file <none>
I20260812 14:51:06.451421 14456 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:06.451470 14456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:06.451644 14456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:06.452817 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "e45c5904d2a4422fb7a050c9a94806d2"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "b2bfecb96c733163b4e62fe42c226fcd"
server_key_iv: "92a73732e3fccbe83a813ceb996d5c0f"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.453272 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "e45c5904d2a4422fb7a050c9a94806d2"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "b2bfecb96c733163b4e62fe42c226fcd"
server_key_iv: "92a73732e3fccbe83a813ceb996d5c0f"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.454736 14456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:06.455551 14479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.455768 14456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:06.455870 14456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "e45c5904d2a4422fb7a050c9a94806d2"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "b2bfecb96c733163b4e62fe42c226fcd"
server_key_iv: "92a73732e3fccbe83a813ceb996d5c0f"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.455946 14456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:06.470124 14456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:06.470503 14456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:06.470687 14456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:06.475157 14456 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:46467
I20260812 14:51:06.475195 14583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:46467 every 8 connection(s)
I20260812 14:51:06.475574 14456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:06.476640 14584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.478766 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14456
I20260812 14:51:06.478855 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:06.479074 13898 external_mini_cluster.cc:1428] Setting key 9895c69346591b499ecc05ce060845e7
I20260812 14:51:06.479486 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: Bootstrap starting.
I20260812 14:51:06.480368 14584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.480741 14584 log.cc:824] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.481717 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: No bootstrap required, opened a new log
I20260812 14:51:06.484045 14584 raft_consensus.cc:348] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } }
I20260812 14:51:06.484248 14584 raft_consensus.cc:374] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.484289 14584 raft_consensus.cc:729] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45c5904d2a4422fb7a050c9a94806d2, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.484462 14584 consensus_queue.cc:260] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } }
I20260812 14:51:06.484534 14584 raft_consensus.cc:388] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:06.484556 14584 raft_consensus.cc:482] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:06.484623 14584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:06.485407 14584 raft_consensus.cc:504] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } }
I20260812 14:51:06.485527 14584 leader_election.cc:302] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e45c5904d2a4422fb7a050c9a94806d2; no voters:
I20260812 14:51:06.485698 14584 leader_election.cc:288] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:06.485772 14594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:06.485809 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.479697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.13.146.190:46467)
0812 14:51:06.479940 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.479952 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.479993 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:51:06.480619 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.480629 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.480736 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.480896 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:06.481165 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.482591 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.483404 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.484104 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.484238 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.485010 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:06.485014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:06.485015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:06.485264 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:06.485310 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:06.485325 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.485401 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.485465 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.485552 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260812 14:51:06.485987 14594 raft_consensus.cc:686] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [term 1 LEADER]: Becoming Leader. State: Replica: e45c5904d2a4422fb7a050c9a94806d2, State: Running, Role: LEADER
I20260812 14:51:06.485991 14584 sys_catalog.cc:564] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:06.486251 14594 consensus_queue.cc:237] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } }
I20260812 14:51:06.487131 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e45c5904d2a4422fb7a050c9a94806d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } } }
I20260812 14:51:06.487221 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:06.487481 14616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:06.487632 14597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e45c5904d2a4422fb7a050c9a94806d2. Latest consensus state: current_term: 1 leader_uuid: "e45c5904d2a4422fb7a050c9a94806d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e45c5904d2a4422fb7a050c9a94806d2" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46467 } } }
I20260812 14:51:06.487833 14597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:06.488106 14616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:06.489638 14616 catalog_manager.cc:1347] Generated new cluster ID: c35810360c09467dba9e44d78780b1ae
I20260812 14:51:06.489699 14616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:06.494400 14616 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:06.495126 14616 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:06.498986 14616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: Generated new TSK 0
I20260812 14:51:06.499163 14616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:06.501997 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--builtin_ntp_servers=127.13.146.148:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:06.588374 14626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:06.588620 14626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:06.588660 14626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:06.589883 14626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:06.589969 14626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:06.590011 14626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:06.592166 14626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:06.592563 14626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:06.592942 14626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:06.602437 14632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.602458 14634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.602819 14636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:06.603273 14626 server_base.cc:1034] running on GCE node
I20260812 14:51:06.603519 14626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:06.603834 14626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:06.606861 14626 hybrid_clock.cc:648] HybridClock initialized: now 1786546266606832 us; error 41 us; skew 500 ppm
I20260812 14:51:06.610536 14626 webserver.cc:460] Webserver started at http://127.13.146.129:39963/ using document root <none> and password file <none>
I20260812 14:51:06.610894 14626 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:06.610975 14626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:06.611140 14626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:06.612790 14626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "2dddd72601e47667b879d4d930e5d094"
server_key_iv: "294843eccc24ba63f86aa5c4c0b66ed9"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.613193 14626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "2dddd72601e47667b879d4d930e5d094"
server_key_iv: "294843eccc24ba63f86aa5c4c0b66ed9"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.614671 14626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:06.615505 14645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.615679 14626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:06.615793 14626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "2dddd72601e47667b879d4d930e5d094"
server_key_iv: "294843eccc24ba63f86aa5c4c0b66ed9"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.615877 14626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:06.624826 14626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:06.625144 14626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:06.625371 14626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:06.625626 14626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:06.626052 14626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:06.626091 14626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.626114 14626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:06.626158 14626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.632689 14626 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:45627
I20260812 14:51:06.632722 14854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:45627 every 8 connection(s)
I20260812 14:51:06.633098 14626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:51:06.638592 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.633981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:33903 (local address 127.13.146.190:46467)
0812 14:51:06.634037 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.634039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.634255 (+ 216us) server_negotiation.cc:407] Connection header received
0812 14:51:06.634859 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.634861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.634871 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.634938 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:06.635629 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.636125 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.637128 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.637394 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.637451 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.638323 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:06.638325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:06.638325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:06.638329 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:06.638345 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:06.638349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.638398 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.638500 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.638524 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:51:06.638779 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.633894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46467 (local address 127.13.146.129:33903)
0812 14:51:06.634092 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.634137 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.634760 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.635010 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.635014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.635022 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:06.635548 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.635552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.636222 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.636222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.637012 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.637014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.637472 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.637473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.637591 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.638158 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:06.638165 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:06.638195 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:06.638218 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:06.638413 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:06.638417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:06.638420 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.638473 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.638558 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260812 14:51:06.639325 14857 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46467
I20260812 14:51:06.639441 14857 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:06.639636 14857 heartbeater.cc:511] Master 127.13.146.190:46467 requested a full tablet report, sending...
I20260812 14:51:06.640237 14515 ts_manager.cc:194] Registered new tserver with Master: ac6c7ce1ac9c410ab29b8889cec3d1ca (127.13.146.129:45627)
I20260812 14:51:06.640914 14515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.129:33903
I20260812 14:51:06.641197 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14626
I20260812 14:51:06.641340 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:06.641542 13898 external_mini_cluster.cc:1428] Setting key 07f7fd0c2bce5c4d9253fef31acffabe
I20260812 14:51:06.642489 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--builtin_ntp_servers=127.13.146.148:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:06.727358 14867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:06.727591 14867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:06.727615 14867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:06.728832 14867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:06.728911 14867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:06.728955 14867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:06.730394 14867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:06.730690 14867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:06.731011 14867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:06.736836 14876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.736828 14874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:06.736920 14867 server_base.cc:1034] running on GCE node
W20260812 14:51:06.736828 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:06.737331 14867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:06.737632 14867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:06.738838 14867 hybrid_clock.cc:648] HybridClock initialized: now 1786546266738833 us; error 37 us; skew 500 ppm
I20260812 14:51:06.740427 14867 webserver.cc:460] Webserver started at http://127.13.146.130:34887/ using document root <none> and password file <none>
I20260812 14:51:06.740706 14867 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:06.740762 14867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:06.740984 14867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:06.742312 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "4644117e2449474cbd46f9f1a0df10df"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "f54f16a28331cd31794634588ef5733a"
server_key_iv: "0f11e0a70e5638a80c74495116224a4e"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.742743 14867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "4644117e2449474cbd46f9f1a0df10df"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "f54f16a28331cd31794634588ef5733a"
server_key_iv: "0f11e0a70e5638a80c74495116224a4e"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.744432 14867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:06.745380 14892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.745563 14867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:06.745666 14867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "4644117e2449474cbd46f9f1a0df10df"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "f54f16a28331cd31794634588ef5733a"
server_key_iv: "0f11e0a70e5638a80c74495116224a4e"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.745731 14867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:06.762496 14867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:06.762799 14867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:06.762910 14867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:06.763144 14867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:06.763461 14867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:06.763494 14867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.763518 14867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:06.763531 14867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.770044 14867 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:41163
I20260812 14:51:06.770097 15106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:41163 every 8 connection(s)
I20260812 14:51:06.770452 14867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:51:06.776052 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.771201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:39773 (local address 127.13.146.190:46467)
0812 14:51:06.771264 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.771267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.771509 (+ 242us) server_negotiation.cc:407] Connection header received
0812 14:51:06.772083 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.772085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.772098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.772175 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:06.772917 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.773530 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.774534 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.774728 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.774795 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.775768 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:06.775769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:06.775770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:06.775776 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:06.775793 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:06.775797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.775861 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.775953 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.775987 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:51:06.776202 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.771161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46467 (local address 127.13.146.130:39773)
0812 14:51:06.771368 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.771412 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.771950 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.772261 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.772265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.772273 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:06.772819 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.772824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.773639 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.773640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.774437 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.774439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.774814 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.774814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.774929 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.775575 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:06.775575 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:06.775607 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:06.775631 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:06.775876 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:06.775881 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:06.775884 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.775940 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.776036 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260812 14:51:06.776801 15107 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46467
I20260812 14:51:06.776925 15107 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:06.777128 15107 heartbeater.cc:511] Master 127.13.146.190:46467 requested a full tablet report, sending...
I20260812 14:51:06.777554 14515 ts_manager.cc:194] Registered new tserver with Master: 4644117e2449474cbd46f9f1a0df10df (127.13.146.130:41163)
I20260812 14:51:06.777956 14515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.130:39773
I20260812 14:51:06.778961 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 14867
I20260812 14:51:06.779089 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:06.779309 13898 external_mini_cluster.cc:1428] Setting key df653c88a91be71b536c1e72a4df5910
I20260812 14:51:06.780236 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--builtin_ntp_servers=127.13.146.148:38767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:06.868359 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:06.868636 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:06.868690 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:06.869968 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:06.870047 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:06.870091 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:06.871485 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:38767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:46467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:06.871795 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:06.872130 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:06.877465 15125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.877499 15123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:06.877488 15121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:06.877660 15110 server_base.cc:1034] running on GCE node
I20260812 14:51:06.877872 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:06.878116 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:06.879289 15110 hybrid_clock.cc:648] HybridClock initialized: now 1786546266879276 us; error 27 us; skew 500 ppm
I20260812 14:51:06.880836 15110 webserver.cc:460] Webserver started at http://127.13.146.131:44201/ using document root <none> and password file <none>
I20260812 14:51:06.881296 15110 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:06.881389 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:06.881579 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:06.883651 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "0bc0f9b6493b4b5494b9400948201db6"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "40ad813ce46ad053f05e4dc376f3473a"
server_key_iv: "864b54d3e47ce378ad186887bcfafd67"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.884192 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "0bc0f9b6493b4b5494b9400948201db6"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "40ad813ce46ad053f05e4dc376f3473a"
server_key_iv: "864b54d3e47ce378ad186887bcfafd67"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.886488 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.005s
I20260812 14:51:06.887890 15135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.888090 15110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:06.888221 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "0bc0f9b6493b4b5494b9400948201db6"
format_stamp: "Formatted at 2026-08-12 14:51:06 on dist-test-slave-8hhm"
server_key: "40ad813ce46ad053f05e4dc376f3473a"
server_key_iv: "864b54d3e47ce378ad186887bcfafd67"
server_key_version: "encryptionkey@0"
I20260812 14:51:06.888311 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:06.897047 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:06.897430 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:06.897590 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:06.897830 15110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:06.898162 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:06.898218 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.898263 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:06.898286 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:06.904376 15110 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:35161
I20260812 14:51:06.904425 15340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:35161 every 8 connection(s)
I20260812 14:51:06.904769 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:51:06.906270 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15110
I20260812 14:51:06.906381 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:06.906599 13898 external_mini_cluster.cc:1428] Setting key 6a87ab16ce40fa79da7467e95cd96d10
I20260812 14:51:06.910728 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.905562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:47207 (local address 127.13.146.190:46467)
0812 14:51:06.905624 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.905626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.905876 (+ 250us) server_negotiation.cc:407] Connection header received
0812 14:51:06.906668 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.906670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.906684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.906757 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:06.907691 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.908248 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.909353 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.909583 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.909651 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.910502 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:06.910503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:06.910503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:06.910507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:06.910523 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:06.910526 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.910576 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.910646 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.910669 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:51:06.910944 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.905538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46467 (local address 127.13.146.131:47207)
0812 14:51:06.905755 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.905789 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.906530 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.906856 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.906859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.906868 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:06.907581 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.907587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.908349 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.908350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.909150 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.909152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.909668 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.909670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.909785 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.910353 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:06.910353 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:06.910381 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:06.910404 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:06.910589 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:06.910593 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:06.910596 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.910649 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.910731 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260812 14:51:06.911386 15341 heartbeater.cc:347] Connected to a master server at 127.13.146.190:46467
I20260812 14:51:06.911477 15341 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:06.911628 15341 heartbeater.cc:511] Master 127.13.146.190:46467 requested a full tablet report, sending...
I20260812 14:51:06.912016 14515 ts_manager.cc:194] Registered new tserver with Master: 0bc0f9b6493b4b5494b9400948201db6 (127.13.146.131:35161)
I20260812 14:51:06.912348 14515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.131:47207
I20260812 14:51:06.918987 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:06.923974 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.920485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.13.146.190:46467)
0812 14:51:06.920596 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.920599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.920656 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:51:06.920754 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.920756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.920769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.920852 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:06.921062 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.921715 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.922605 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.922823 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.922905 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.923708 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:06.923710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:06.923711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:06.923716 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:06.923731 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:06.923735 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:06.923783 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.923854 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.923877 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 14:51:06.925741 14515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:06.927456 14515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:06.934485 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.929959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49862 (local address 127.13.146.129:45627)
0812 14:51:06.930275 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.930280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.930294 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:51:06.930759 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.930762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.930779 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.931003 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.931086 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.931831 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.933180 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.934037 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.934144 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.934247 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.934344 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.934388 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":109,"threads_started":1}
I20260812 14:51:06.934474 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.929875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:45627 (local address 127.0.0.1:49862)
0812 14:51:06.930045 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.930142 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.930225 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.930881 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.930881 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.930883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.930930 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.930932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.931926 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.931927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.933082 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.933085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.934145 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.934146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.934229 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.934243 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.934329 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.934388 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260812 14:51:06.934669 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.929876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:41163 (local address 127.0.0.1:56318)
0812 14:51:06.930202 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.930213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.930267 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.930655 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.930658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.930661 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.930738 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.930740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.932365 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.932366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.933430 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.933433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.934314 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.934315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.934377 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.934381 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.934431 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.934472 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":80,"thread_start_us":94,"threads_started":1}
I20260812 14:51:06.935096 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.930214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.13.146.130:41163)
0812 14:51:06.930412 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.930416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.930424 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:06.930459 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.930460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.930474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.930582 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.930908 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.932260 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.933519 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.934112 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.934196 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.934277 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.934519 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.934619 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260812 14:51:06.935320 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.930428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.13.146.131:35161)
0812 14:51:06.930589 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:06.930594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:06.930796 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:51:06.930946 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:06.930950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:06.930968 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:06.931059 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:06.931184 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.932399 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.933899 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.934616 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.934701 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.934784 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:06.935184 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 14:51:06.935231 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260812 14:51:06.935375 14734 tablet_service.cc:1467] Processing CreateTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 (DEFAULT_TABLE table=test-table [id=9ef055e4679c42f5802dbd1097e7a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.935621 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:06.930129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:35161 (local address 127.0.0.1:42756)
0812 14:51:06.930732 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:06.930743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:06.930826 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:06.931068 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:06.931069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:06.931071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:06.931103 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.931105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.932507 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.932509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.933794 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:06.933797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:06.935030 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:06.935031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:06.935097 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:06.935102 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:51:06.935165 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:51:06.935555 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":545,"threads_started":1}
I20260812 14:51:06.935856 14734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2393d67077ce4856ad0c2bfa1de5ed35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.935806 14986 tablet_service.cc:1467] Processing CreateTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 (DEFAULT_TABLE table=test-table [id=9ef055e4679c42f5802dbd1097e7a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.936177 14986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2393d67077ce4856ad0c2bfa1de5ed35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.936311 15224 tablet_service.cc:1467] Processing CreateTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 (DEFAULT_TABLE table=test-table [id=9ef055e4679c42f5802dbd1097e7a0c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:06.936653 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2393d67077ce4856ad0c2bfa1de5ed35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:06.939039 15359 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Bootstrap starting.
I20260812 14:51:06.939854 15359 tablet_bootstrap.cc:654] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.940186 15361 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Bootstrap starting.
I20260812 14:51:06.940225 15362 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Bootstrap starting.
I20260812 14:51:06.940271 15359 log.cc:824] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.940874 15362 tablet_bootstrap.cc:654] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.940874 15361 tablet_bootstrap.cc:654] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:06.941171 15359 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: No bootstrap required, opened a new log
I20260812 14:51:06.941253 15362 log.cc:824] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.941267 15361 log.cc:824] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:06.941289 15359 ts_tablet_manager.cc:1397] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 14:51:06.942044 15362 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: No bootstrap required, opened a new log
I20260812 14:51:06.942121 15362 ts_tablet_manager.cc:1397] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:51:06.942394 15361 tablet_bootstrap.cc:492] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: No bootstrap required, opened a new log
I20260812 14:51:06.942459 15361 ts_tablet_manager.cc:1397] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:51:06.943778 15359 raft_consensus.cc:348] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.943954 15359 raft_consensus.cc:374] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.944003 15359 raft_consensus.cc:729] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6c7ce1ac9c410ab29b8889cec3d1ca, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.944204 15359 consensus_queue.cc:260] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.944541 15359 ts_tablet_manager.cc:1428] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:51:06.944571 14857 heartbeater.cc:503] Master 127.13.146.190:46467 was elected leader, sending a full tablet report...
I20260812 14:51:06.944808 15362 raft_consensus.cc:348] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.944963 15362 raft_consensus.cc:374] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.945004 15362 raft_consensus.cc:729] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc0f9b6493b4b5494b9400948201db6, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.945140 15362 consensus_queue.cc:260] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.945420 15362 ts_tablet_manager.cc:1428] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 14:51:06.945461 15341 heartbeater.cc:503] Master 127.13.146.190:46467 was elected leader, sending a full tablet report...
I20260812 14:51:06.945484 15361 raft_consensus.cc:348] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.945655 15361 raft_consensus.cc:374] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:06.945693 15361 raft_consensus.cc:729] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4644117e2449474cbd46f9f1a0df10df, State: Initialized, Role: FOLLOWER
I20260812 14:51:06.945829 15361 consensus_queue.cc:260] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:06.946096 15361 ts_tablet_manager.cc:1428] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 14:51:06.946233 15107 heartbeater.cc:503] Master 127.13.146.190:46467 was elected leader, sending a full tablet report...
W20260812 14:51:07.021026 15108 tablet.cc:2369] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:51:07.133958 14859 tablet.cc:2369] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:51:07.155349 15343 tablet.cc:2369] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:07.177075 15372 raft_consensus.cc:482] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:07.177273 15372 raft_consensus.cc:504] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:07.177768 15372 leader_election.cc:288] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac6c7ce1ac9c410ab29b8889cec3d1ca (127.13.146.129:45627), 0bc0f9b6493b4b5494b9400948201db6 (127.13.146.131:35161)
I20260812 14:51:07.180774 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.178176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:49245 (local address 127.13.146.131:35161)
0812 14:51:07.178234 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:07.178236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:07.178283 (+ 47us) server_negotiation.cc:407] Connection header received
0812 14:51:07.178351 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:07.178352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:07.178361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:07.178413 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:07.178577 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.179041 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180044 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.180413 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180504 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:07.180512 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:07.180669 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 14:51:07.180706 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:51:07.180788 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.178054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:35161 (local address 127.13.146.130:49245)
0812 14:51:07.178236 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:07.178245 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:07.178295 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:07.178415 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:07.178416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:07.178417 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:07.178487 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:07.178488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.179132 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.179133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:07.179944 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:07.179946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.180504 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:07.180576 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:07.180586 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:51:07.180656 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:51:07.180697 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260812 14:51:07.180917 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.178050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:34479 (local address 127.13.146.129:45627)
0812 14:51:07.178166 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:07.178168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:07.178182 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:51:07.178218 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:07.178219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:07.178235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:07.178335 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:07.178525 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.179068 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180123 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.180530 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180624 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:07.180634 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:07.180803 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:51:07.180844 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 14:51:07.180919 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.177967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:45627 (local address 127.13.146.130:34479)
0812 14:51:07.178028 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:07.178041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:07.178086 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:07.178343 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:07.178345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:07.178347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:07.178429 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:07.178431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.179172 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.179174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:07.179999 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:07.180002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.180640 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.180642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:07.180716 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:07.180719 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:07.180794 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:51:07.180838 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":45}
I20260812 14:51:07.181205 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2393d67077ce4856ad0c2bfa1de5ed35" candidate_uuid: "4644117e2449474cbd46f9f1a0df10df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc0f9b6493b4b5494b9400948201db6" is_pre_election: true
I20260812 14:51:07.181305 14774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2393d67077ce4856ad0c2bfa1de5ed35" candidate_uuid: "4644117e2449474cbd46f9f1a0df10df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" is_pre_election: true
I20260812 14:51:07.181433 15267 raft_consensus.cc:2454] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4644117e2449474cbd46f9f1a0df10df in term 0.
I20260812 14:51:07.181478 14774 raft_consensus.cc:2454] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4644117e2449474cbd46f9f1a0df10df in term 0.
I20260812 14:51:07.181706 14893 leader_election.cc:302] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc0f9b6493b4b5494b9400948201db6, 4644117e2449474cbd46f9f1a0df10df; no voters:
I20260812 14:51:07.181885 15372 raft_consensus.cc:2781] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:07.181957 15372 raft_consensus.cc:482] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:07.181991 15372 raft_consensus.cc:3037] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:07.182760 15372 raft_consensus.cc:504] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:07.182927 15372 leader_election.cc:288] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [CANDIDATE]: Term 1 election: Requested vote from peers ac6c7ce1ac9c410ab29b8889cec3d1ca (127.13.146.129:45627), 0bc0f9b6493b4b5494b9400948201db6 (127.13.146.131:35161)
I20260812 14:51:07.183095 14774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2393d67077ce4856ad0c2bfa1de5ed35" candidate_uuid: "4644117e2449474cbd46f9f1a0df10df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca"
I20260812 14:51:07.183185 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2393d67077ce4856ad0c2bfa1de5ed35" candidate_uuid: "4644117e2449474cbd46f9f1a0df10df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc0f9b6493b4b5494b9400948201db6"
I20260812 14:51:07.183223 14774 raft_consensus.cc:3037] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:07.183284 15267 raft_consensus.cc:3037] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:07.184077 15267 raft_consensus.cc:2454] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4644117e2449474cbd46f9f1a0df10df in term 1.
I20260812 14:51:07.184080 14774 raft_consensus.cc:2454] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4644117e2449474cbd46f9f1a0df10df in term 1.
I20260812 14:51:07.184262 14893 leader_election.cc:302] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc0f9b6493b4b5494b9400948201db6, 4644117e2449474cbd46f9f1a0df10df; no voters:
I20260812 14:51:07.184368 15372 raft_consensus.cc:2781] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:07.184522 15372 raft_consensus.cc:686] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 1 LEADER]: Becoming Leader. State: Replica: 4644117e2449474cbd46f9f1a0df10df, State: Running, Role: LEADER
I20260812 14:51:07.184626 15372 consensus_queue.cc:237] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } }
I20260812 14:51:07.185531 14509 catalog_manager.cc:5495] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df reported cstate change: term changed from 0 to 1, leader changed from <none> to 4644117e2449474cbd46f9f1a0df10df (127.13.146.130). New cstate: current_term: 1 leader_uuid: "4644117e2449474cbd46f9f1a0df10df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4644117e2449474cbd46f9f1a0df10df" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 } health_report { overall_health: UNKNOWN } } }
I20260812 14:51:07.222966 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.220469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.13.146.130:41163)
0812 14:51:07.220537 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:07.220539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:07.220550 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:51:07.220599 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:07.220599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:07.220620 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:07.220727 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:07.220868 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.221347 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.222220 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:07.222401 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:07.222451 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:07.222611 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:07.222716 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:07.222775 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:07.222819 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 14:51:07.222837 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:51:07.224943 15267 raft_consensus.cc:1261] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 1 FOLLOWER]: Refusing update from remote peer 4644117e2449474cbd46f9f1a0df10df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:07.224953 14774 raft_consensus.cc:1261] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 1 FOLLOWER]: Refusing update from remote peer 4644117e2449474cbd46f9f1a0df10df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:07.225371 15372 consensus_queue.cc:1035] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc0f9b6493b4b5494b9400948201db6" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:07.225582 15375 consensus_queue.cc:1035] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac6c7ce1ac9c410ab29b8889cec3d1ca" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 45627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:07.228658 15389 mvcc.cc:205] Tried to move back new op lower bound from 7317693510550401024 to 7317693510388981760. Current Snapshot: MvccSnapshot[applied={T|T < 7317693510550401024}]
I20260812 14:51:07.230589 14513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:07.230679 14513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:07.231503 14513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e45c5904d2a4422fb7a050c9a94806d2: Sending DeleteTablet for 3 replicas of tablet 2393d67077ce4856ad0c2bfa1de5ed35
I20260812 14:51:07.231874 14986 tablet_service.cc:1514] Processing DeleteTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:07 UTC) from {username='slave'} at 127.0.0.1:56318
I20260812 14:51:07.231889 14734 tablet_service.cc:1514] Processing DeleteTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:07 UTC) from {username='slave'} at 127.0.0.1:49862
I20260812 14:51:07.232134 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet 2393d67077ce4856ad0c2bfa1de5ed35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:07 UTC) from {username='slave'} at 127.0.0.1:42756
I20260812 14:51:07.232180 15407 tablet_replica.cc:330] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: stopping tablet replica
I20260812 14:51:07.232268 15407 raft_consensus.cc:2229] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:51:07.232372 15407 raft_consensus.cc:2258] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:07.232390 15408 tablet_replica.cc:330] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: stopping tablet replica
I20260812 14:51:07.232483 15408 raft_consensus.cc:2229] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:07.232770 15404 tablet_replica.cc:330] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca: stopping tablet replica
I20260812 14:51:07.232827 15407 ts_tablet_manager.cc:1905] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:07.232860 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14626
I20260812 14:51:07.232926 15404 raft_consensus.cc:2229] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:07.233081 15404 raft_consensus.cc:2258] T 2393d67077ce4856ad0c2bfa1de5ed35 P ac6c7ce1ac9c410ab29b8889cec3d1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:07.235263 15407 ts_tablet_manager.cc:1918] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:07.235364 15407 log.cc:1196] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/2393d67077ce4856ad0c2bfa1de5ed35
I20260812 14:51:07.235937 15407 ts_tablet_manager.cc:1939] T 2393d67077ce4856ad0c2bfa1de5ed35 P 4644117e2449474cbd46f9f1a0df10df: Deleting consensus metadata
I20260812 14:51:07.236495 14480 catalog_manager.cc:4841] TS 4644117e2449474cbd46f9f1a0df10df (127.13.146.130:41163): tablet 2393d67077ce4856ad0c2bfa1de5ed35 (table test-table [id=9ef055e4679c42f5802dbd1097e7a0c6]) successfully deleted
I20260812 14:51:07.273967 15408 raft_consensus.cc:2258] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:07.275450 15408 ts_tablet_manager.cc:1905] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:07.277377 15408 ts_tablet_manager.cc:1918] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:07.277482 15408 log.cc:1196] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/2393d67077ce4856ad0c2bfa1de5ed35
I20260812 14:51:07.277827 15408 ts_tablet_manager.cc:1939] T 2393d67077ce4856ad0c2bfa1de5ed35 P 0bc0f9b6493b4b5494b9400948201db6: Deleting consensus metadata
I20260812 14:51:07.278424 14480 catalog_manager.cc:4841] TS 0bc0f9b6493b4b5494b9400948201db6 (127.13.146.131:35161): tablet 2393d67077ce4856ad0c2bfa1de5ed35 (table test-table [id=9ef055e4679c42f5802dbd1097e7a0c6]) successfully deleted
W20260812 14:51:07.280531 14484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:45627 (error 108)
I20260812 14:51:07.280798 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14867
I20260812 14:51:07.284049 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:07.280925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:45627 (local address 127.0.0.1:49872)
0812 14:51:07.283933 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:07.284004 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:45627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2978}
W20260812 14:51:07.284251 14484 catalog_manager.cc:4584] TS ac6c7ce1ac9c410ab29b8889cec3d1ca (127.13.146.129:45627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2393d67077ce4856ad0c2bfa1de5ed35: Network error: Client connection negotiation failed: client connection to 127.13.146.129:45627: connect: Connection refused (error 111)
I20260812 14:51:07.292111 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15110
I20260812 14:51:07.307159 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 14456
2026-08-12T14:51:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (996 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:07 dist-test-slave-8hhm krb5kdc[15412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:07 dist-test-slave-8hhm krb5kdc[15412](info): set up 2 sockets
Aug 12 14:51:07 dist-test-slave-8hhm krb5kdc[15412](info): commencing operation
krb5kdc: starting...
W20260812 14:51:08.636653 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:08 dist-test-slave-8hhm krb5kdc[15412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:08.767872 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36241
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:36241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:08.852478 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:08.852695 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:08.852718 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:08.854007 15441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:08.854056 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:08.854070 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:08.854084 15441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:08.854095 15441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:08.855620 15441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:36241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:08.855888 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:08.856138 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:08.861243 15451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:08.861263 15450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:08.861446 15441 server_base.cc:1034] running on GCE node
W20260812 14:51:08.861315 15454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:08.861850 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:08.862156 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:08.863355 15441 hybrid_clock.cc:648] HybridClock initialized: now 1786546268863348 us; error 55 us; skew 500 ppm
Aug 12 14:51:08 dist-test-slave-8hhm krb5kdc[15412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546268, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:08.865995 15441 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:08.866501 15441 webserver.cc:460] Webserver started at http://127.13.146.190:44011/ using document root <none> and password file <none>
I20260812 14:51:08.866770 15441 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:08.866844 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:08.867003 15441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:08.868112 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "54bb1c7cef3a4fb9b73ab80347c661e4"
format_stamp: "Formatted at 2026-08-12 14:51:08 on dist-test-slave-8hhm"
server_key: "bf0d28c50fe03b18107b28744ff34710"
server_key_iv: "07976ec8eb92cc90874e990fcfa332b1"
server_key_version: "encryptionkey@0"
I20260812 14:51:08.868520 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "54bb1c7cef3a4fb9b73ab80347c661e4"
format_stamp: "Formatted at 2026-08-12 14:51:08 on dist-test-slave-8hhm"
server_key: "bf0d28c50fe03b18107b28744ff34710"
server_key_iv: "07976ec8eb92cc90874e990fcfa332b1"
server_key_version: "encryptionkey@0"
I20260812 14:51:08.870093 15441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:51:08.870891 15465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:08.871120 15441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:08.871243 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "54bb1c7cef3a4fb9b73ab80347c661e4"
format_stamp: "Formatted at 2026-08-12 14:51:08 on dist-test-slave-8hhm"
server_key: "bf0d28c50fe03b18107b28744ff34710"
server_key_iv: "07976ec8eb92cc90874e990fcfa332b1"
server_key_version: "encryptionkey@0"
I20260812 14:51:08.871328 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:08.885469 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:08.888566 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:08.888754 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:08.893465 15441 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:36241
I20260812 14:51:08.893515 15555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:36241 every 8 connection(s)
I20260812 14:51:08.893949 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:08.894927 15556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:08.897372 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: Bootstrap starting.
I20260812 14:51:08.897946 15556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:08.898201 15556 log.cc:824] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:08.898917 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: No bootstrap required, opened a new log
I20260812 14:51:08.900480 15556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } }
I20260812 14:51:08.900625 15556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:08.900648 15556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54bb1c7cef3a4fb9b73ab80347c661e4, State: Initialized, Role: FOLLOWER
I20260812 14:51:08.900799 15556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } }
I20260812 14:51:08.900883 15556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:08.900907 15556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:08.900938 15556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:08.901530 15556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } }
I20260812 14:51:08.901621 15556 leader_election.cc:302] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54bb1c7cef3a4fb9b73ab80347c661e4; no voters:
I20260812 14:51:08.901777 15556 leader_election.cc:288] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:08.901846 15561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:08.902016 15561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [term 1 LEADER]: Becoming Leader. State: Replica: 54bb1c7cef3a4fb9b73ab80347c661e4, State: Running, Role: LEADER
I20260812 14:51:08.902055 15556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:08.902110 15561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } }
I20260812 14:51:08.902490 15568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54bb1c7cef3a4fb9b73ab80347c661e4. Latest consensus state: current_term: 1 leader_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } } }
I20260812 14:51:08.902575 15568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:08.902630 15566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bb1c7cef3a4fb9b73ab80347c661e4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36241 } } }
I20260812 14:51:08.902693 15566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:08.903172 15574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:08.903786 15574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:08.904083 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15441
I20260812 14:51:08.904207 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:08.904444 13898 external_mini_cluster.cc:1428] Setting key 952702ef25ca11323a51025e65d96d3a
I20260812 14:51:08.905508 15574 catalog_manager.cc:1347] Generated new cluster ID: 7d34eed75ec1473da2b74267697b4a9c
I20260812 14:51:08.905573 15574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:08 dist-test-slave-8hhm krb5kdc[15412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:08.919270 15574 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:08.919822 15574 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:08.921204 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:08.904915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.13.146.190:36241)
0812 14:51:08.909364 (+ 4449us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:08.909376 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:08.909417 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:51:08.909549 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:08.909556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:08.909627 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:08.909804 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:08.910762 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:08.912187 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:08.913344 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:08.913988 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:08.914091 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:08.917133 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:08.917138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:08.917139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:08.917153 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:08.919377 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:08.919829 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:08.919832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:08.919833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:08.919960 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:08.920194 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:08.920195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:08.920195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:08.920486 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:08.920616 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:08.920799 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 14:51:08.920955 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":4387,"thread_start_us":203,"threads_started":1}
I20260812 14:51:08.927640 15574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: Generated new TSK 0
I20260812 14:51:08.927867 15574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:51:08.960206 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:09.051196 15598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:09.051424 15598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:09.051448 15598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:09.052654 15598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:09.052701 15598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:09.052728 15598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:09.054489 15598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:09.054792 15598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:09.055042 15598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:09.061110 15617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.061121 15612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.061158 15598 server_base.cc:1034] running on GCE node
W20260812 14:51:09.061125 15613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.061635 15598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:09.061919 15598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:09.063093 15598 hybrid_clock.cc:648] HybridClock initialized: now 1786546269063090 us; error 38 us; skew 500 ppm
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:09.065943 15598 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:51:09.066507 15598 webserver.cc:460] Webserver started at http://127.13.146.129:39265/ using document root <none> and password file <none>
I20260812 14:51:09.066794 15598 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:09.066854 15598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:09.067013 15598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:09.068154 15598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "243eb887d62f4fc385a7b5f77c4734d8"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "384de6a343c8b1eef89c49385c919995"
server_key_iv: "c1e297a80c9fe91c4026077486e5ff84"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.068547 15598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "243eb887d62f4fc385a7b5f77c4734d8"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "384de6a343c8b1eef89c49385c919995"
server_key_iv: "c1e297a80c9fe91c4026077486e5ff84"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.070106 15598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:09.070917 15627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.071146 15598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:09.071252 15598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "243eb887d62f4fc385a7b5f77c4734d8"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "384de6a343c8b1eef89c49385c919995"
server_key_iv: "c1e297a80c9fe91c4026077486e5ff84"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.071338 15598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:09.078961 15598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:09.081573 15598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:09.081734 15598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:09.081997 15598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:09.082340 15598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:09.082384 15598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.082429 15598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:09.082458 15598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.088822 15598 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:43961
I20260812 14:51:09.088904 15831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:43961 every 8 connection(s)
I20260812 14:51:09.089380 15598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:09.096897 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15598
I20260812 14:51:09.096997 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:09.097311 13898 external_mini_cluster.cc:1428] Setting key 1267cc8969e29bc4d2b6631276bbb3bf
I20260812 14:51:09.100199 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.090613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:52719 (local address 127.13.146.190:36241)
0812 14:51:09.090674 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.090676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.090688 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:51:09.091518 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.091520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.091534 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.091594 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.092637 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.093165 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.094294 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.094516 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.094593 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.096872 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.096874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.096874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:09.096882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.098648 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.099104 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.099105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.099106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.099205 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.099486 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.099487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.099487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.099612 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.099687 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.099979 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.100090 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:51:09.100373 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.090280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36241 (local address 127.13.146.129:52719)
0812 14:51:09.090481 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.090544 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.091395 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.091673 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.091678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.092307 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:09.092508 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.092514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.093360 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.093361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.094185 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.094187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.094602 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.094603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.094744 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.095255 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:09.095264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:09.096760 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:09.098757 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.098765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.098768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.099012 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.099285 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.099286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.099286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.099409 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.099691 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:09.099698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:09.099845 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.099957 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.100126 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260812 14:51:09.100931 15832 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36241
I20260812 14:51:09.101049 15832 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:09.101292 15832 heartbeater.cc:511] Master 127.13.146.190:36241 requested a full tablet report, sending...
I20260812 14:51:09.101913 15494 ts_manager.cc:194] Registered new tserver with Master: 243eb887d62f4fc385a7b5f77c4734d8 (127.13.146.129:43961)
I20260812 14:51:09.102861 15494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:52719
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:51:09.127942 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:09.213171 15839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:09.213500 15839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:09.213543 15839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:09.214680 15839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:09.214754 15839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:09.214798 15839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:09.216259 15839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:09.216540 15839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:09.216786 15839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:09.221968 15857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.222182 15839 server_base.cc:1034] running on GCE node
W20260812 14:51:09.221968 15853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.222016 15854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.222470 15839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:09.222702 15839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:09.223846 15839 hybrid_clock.cc:648] HybridClock initialized: now 1786546269223837 us; error 35 us; skew 500 ppm
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:09.226467 15839 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:51:09.226956 15839 webserver.cc:460] Webserver started at http://127.13.146.130:46025/ using document root <none> and password file <none>
I20260812 14:51:09.227180 15839 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:09.227222 15839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:09.227317 15839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:09.228353 15839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "7e45c988f1664b9db6bb640f2400b492"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "f506ccd8e16f2b21633d0b0d90e0cdbb"
server_key_iv: "be8d09ab62bac372b8e0f10681b5b896"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.228672 15839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "7e45c988f1664b9db6bb640f2400b492"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "f506ccd8e16f2b21633d0b0d90e0cdbb"
server_key_iv: "be8d09ab62bac372b8e0f10681b5b896"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.230140 15839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:09.230873 15869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.231074 15839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:09.231172 15839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "7e45c988f1664b9db6bb640f2400b492"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "f506ccd8e16f2b21633d0b0d90e0cdbb"
server_key_iv: "be8d09ab62bac372b8e0f10681b5b896"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.231231 15839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:09.257582 15839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:09.259642 15839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:09.259759 15839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:09.259956 15839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:09.260303 15839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:09.260340 15839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.260388 15839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:09.260416 15839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.266360 15839 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:35091
I20260812 14:51:09.266446 16073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:35091 every 8 connection(s)
I20260812 14:51:09.266768 15839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:09.274415 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 15839
I20260812 14:51:09.274535 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:09.274828 13898 external_mini_cluster.cc:1428] Setting key df2ce6f2cb45010b49172127bacae791
I20260812 14:51:09.276774 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.267525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:52979 (local address 127.13.146.190:36241)
0812 14:51:09.267572 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.267575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.267757 (+ 182us) server_negotiation.cc:407] Connection header received
0812 14:51:09.268323 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.268324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.268337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.268404 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.269450 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.269978 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.271008 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.271232 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.271322 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.273591 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.273592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.273593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:09.273609 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.275264 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.275766 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.275768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.275768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.275830 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.276108 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.276109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.276110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.276231 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.276312 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.276597 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.276683 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:51:09.276983 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.267475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36241 (local address 127.13.146.130:52979)
0812 14:51:09.267622 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.267672 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.268212 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.268477 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.268481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.269068 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:09.269351 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.269357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.270086 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.270087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.270904 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.270907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.271339 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.271339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.271480 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.271962 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:09.271970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:09.273497 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:09.275379 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.275386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.275387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.275686 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.275916 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.275917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.275917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.276034 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.276303 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:09.276311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:09.276456 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.276563 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.276732 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260812 14:51:09.277478 16076 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36241
I20260812 14:51:09.277575 16076 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:09.277778 16076 heartbeater.cc:511] Master 127.13.146.190:36241 requested a full tablet report, sending...
I20260812 14:51:09.278205 15494 ts_manager.cc:194] Registered new tserver with Master: 7e45c988f1664b9db6bb640f2400b492 (127.13.146.130:35091)
I20260812 14:51:09.278702 15494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:52979
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:51:09.304250 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:09.387596 16086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:09.387817 16086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:09.387841 16086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:09.389060 16086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:09.389106 16086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:09.389132 16086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:09.390666 16086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:09.390947 16086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:09.391201 16086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:09.396953 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.396953 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.397084 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.397312 16086 server_base.cc:1034] running on GCE node
I20260812 14:51:09.397521 16086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:09.397738 16086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:09.398891 16086 hybrid_clock.cc:648] HybridClock initialized: now 1786546269398873 us; error 30 us; skew 500 ppm
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:09.401585 16086 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:51:09.401980 16086 webserver.cc:460] Webserver started at http://127.13.146.131:34837/ using document root <none> and password file <none>
I20260812 14:51:09.402345 16086 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:09.402441 16086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:09.402580 16086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:09.403802 16086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "2b12020f0d1d465786fd1b35df234193"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "fe2babdc97ed436300330f32cef330fb"
server_key_iv: "41b29b673463eb46c6185063b8af8f3d"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.404198 16086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "2b12020f0d1d465786fd1b35df234193"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "fe2babdc97ed436300330f32cef330fb"
server_key_iv: "41b29b673463eb46c6185063b8af8f3d"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.405618 16086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:09.406380 16111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.406560 16086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:51:09.406664 16086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "2b12020f0d1d465786fd1b35df234193"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "fe2babdc97ed436300330f32cef330fb"
server_key_iv: "41b29b673463eb46c6185063b8af8f3d"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.406750 16086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:09.446350 16086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:09.448931 16086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:09.449112 16086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:09.449406 16086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:09.449743 16086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:09.449798 16086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.449843 16086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:09.449872 16086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.455854 16086 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:35965
I20260812 14:51:09.455937 16298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:35965 every 8 connection(s)
I20260812 14:51:09.456287 16086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:51:09.460944 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16086
I20260812 14:51:09.461090 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:09.461328 13898 external_mini_cluster.cc:1428] Setting key d40181f6bdc769492a192518e4d91ad1
Aug 12 14:51:09 dist-test-slave-8hhm krb5kdc[15412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546269, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:09.466082 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.457137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:60375 (local address 127.13.146.190:36241)
0812 14:51:09.457205 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.457231 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.457364 (+ 133us) server_negotiation.cc:407] Connection header received
0812 14:51:09.457882 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.457883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.457909 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.457982 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.458961 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.459459 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.460433 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.460635 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.460698 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.463021 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.463023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.463023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:09.463030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.464524 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.464976 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.464977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.464978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.465047 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.465378 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.465379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.465380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.465508 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.465588 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.465893 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.465993 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:51:09.466279 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.457059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36241 (local address 127.13.146.131:60375)
0812 14:51:09.457244 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.457289 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.457791 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.458063 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.458068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.458676 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:09.458873 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.458879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.459542 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.459543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.460340 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.460342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.460719 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.460720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.460820 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.461499 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:09.461507 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:09.462922 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:09.464641 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.464649 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.464651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.464895 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.465124 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:09.465125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:09.465125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:09.465303 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:09.465594 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:09.465601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:09.465741 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.465855 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.466033 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":101,"threads_started":1}
I20260812 14:51:09.466877 16301 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36241
I20260812 14:51:09.466966 16301 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:09.467163 16301 heartbeater.cc:511] Master 127.13.146.190:36241 requested a full tablet report, sending...
I20260812 14:51:09.467572 15494 ts_manager.cc:194] Registered new tserver with Master: 2b12020f0d1d465786fd1b35df234193 (127.13.146.131:35965)
I20260812 14:51:09.467906 15494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:60375
I20260812 14:51:09.473265 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:09.482507 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.474932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.13.146.190:36241)
0812 14:51:09.475016 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.475018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.475129 (+ 111us) server_negotiation.cc:407] Connection header received
0812 14:51:09.475237 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.475238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.475249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.475308 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.475975 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.476529 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.477446 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.477667 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.477779 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.479403 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.479404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.479405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:09.479413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.481068 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.481556 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.481558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.481558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.481625 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:09.481881 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:09.481882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:09.481883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:09.482040 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.482175 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.482313 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.482387 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:51:09.484375 15494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:09.486194 15494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:09.492949 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.488956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.13.146.129:43961)
0812 14:51:09.489192 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.489195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.489398 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:51:09.489443 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.489445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.489460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.489537 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:09.489687 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.490440 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.491939 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492491 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492585 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492647 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.492826 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.492870 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260812 14:51:09.492936 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.488746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43961 (local address 127.0.0.1:57798)
0812 14:51:09.489264 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.489274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.489334 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.489536 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.489537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.489539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:09.489595 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.489597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.490551 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.490552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.491829 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.491831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492583 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.492685 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492700 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.492793 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.492851 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260812 14:51:09.493155 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.488746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:35965 (local address 127.0.0.1:55098)
0812 14:51:09.489029 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.489076 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.489146 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.489414 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.489417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.489419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:09.489507 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.489510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.490878 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.490879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.491944 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.491946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492891 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.492955 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492963 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.493015 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.493059 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":8,"thread_start_us":82,"threads_started":1}
I20260812 14:51:09.493285 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.489016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.13.146.130:35091)
0812 14:51:09.489271 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.489276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.489328 (+ 52us) server_negotiation.cc:407] Connection header received
0812 14:51:09.489644 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.489655 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.489682 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.489757 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:09.489870 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.490918 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492068 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492762 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492822 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492868 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.493104 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.493151 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
I20260812 14:51:09.493467 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.488955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.13.146.131:35965)
0812 14:51:09.489159 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.489164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.489178 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:51:09.489248 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.489250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.489267 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.489344 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:09.489596 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.490769 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492017 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492718 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492797 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492852 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.493353 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.493401 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260812 14:51:09.493479 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.488767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:35091 (local address 127.0.0.1:58046)
0812 14:51:09.489272 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.489281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.489330 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.489769 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.489770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.489772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:09.489805 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.489806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.491025 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.491025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.491987 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.491990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.492899 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.492899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.492955 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.492967 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.493015 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.493058 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260812 14:51:09.494093 15710 tablet_service.cc:1467] Processing CreateTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 (DEFAULT_TABLE table=test-table [id=368814bb2dfb4bd6b66d620808d039a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:09.494093 16199 tablet_service.cc:1467] Processing CreateTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 (DEFAULT_TABLE table=test-table [id=368814bb2dfb4bd6b66d620808d039a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:09.494328 15957 tablet_service.cc:1467] Processing CreateTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 (DEFAULT_TABLE table=test-table [id=368814bb2dfb4bd6b66d620808d039a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:09.494477 15710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d54bfbaaa44bcb88b8c90ae9295001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:09.494477 16199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d54bfbaaa44bcb88b8c90ae9295001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:09.494699 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d54bfbaaa44bcb88b8c90ae9295001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:09.497515 16331 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Bootstrap starting.
I20260812 14:51:09.498059 16334 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Bootstrap starting.
I20260812 14:51:09.498351 16331 tablet_bootstrap.cc:654] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:09.498663 16334 tablet_bootstrap.cc:654] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:09.498926 16331 log.cc:824] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:09.499226 16334 log.cc:824] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:09.499240 16332 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: Bootstrap starting.
I20260812 14:51:09.499765 16331 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: No bootstrap required, opened a new log
I20260812 14:51:09.499848 16331 ts_tablet_manager.cc:1397] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:51:09.499908 16332 tablet_bootstrap.cc:654] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:09.499939 16334 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: No bootstrap required, opened a new log
I20260812 14:51:09.500007 16334 ts_tablet_manager.cc:1397] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:51:09.500497 16332 log.cc:824] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:09.501327 16332 tablet_bootstrap.cc:492] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: No bootstrap required, opened a new log
I20260812 14:51:09.501401 16332 ts_tablet_manager.cc:1397] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:51:09.501888 16334 raft_consensus.cc:348] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.502055 16334 raft_consensus.cc:374] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:09.502100 16334 raft_consensus.cc:729] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b12020f0d1d465786fd1b35df234193, State: Initialized, Role: FOLLOWER
I20260812 14:51:09.502048 16331 raft_consensus.cc:348] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.502242 16331 raft_consensus.cc:374] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:09.502224 16334 consensus_queue.cc:260] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.502295 16331 raft_consensus.cc:729] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e45c988f1664b9db6bb640f2400b492, State: Initialized, Role: FOLLOWER
I20260812 14:51:09.502447 16331 consensus_queue.cc:260] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.502501 16334 ts_tablet_manager.cc:1428] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:51:09.502619 16301 heartbeater.cc:503] Master 127.13.146.190:36241 was elected leader, sending a full tablet report...
I20260812 14:51:09.502741 16331 ts_tablet_manager.cc:1428] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:51:09.502923 16076 heartbeater.cc:503] Master 127.13.146.190:36241 was elected leader, sending a full tablet report...
I20260812 14:51:09.503635 16332 raft_consensus.cc:348] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.503852 16332 raft_consensus.cc:374] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:09.503907 16332 raft_consensus.cc:729] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243eb887d62f4fc385a7b5f77c4734d8, State: Initialized, Role: FOLLOWER
I20260812 14:51:09.504019 16332 consensus_queue.cc:260] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.504254 16332 ts_tablet_manager.cc:1428] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:51:09.504326 15832 heartbeater.cc:503] Master 127.13.146.190:36241 was elected leader, sending a full tablet report...
I20260812 14:51:09.506683 16343 raft_consensus.cc:482] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:09.506793 16343 raft_consensus.cc:504] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.507112 16343 leader_election.cc:288] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b12020f0d1d465786fd1b35df234193 (127.13.146.131:35965), 243eb887d62f4fc385a7b5f77c4734d8 (127.13.146.129:43961)
I20260812 14:51:09.511008 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.507318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:35907 (local address 127.13.146.129:43961)
0812 14:51:09.507374 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.507377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.507517 (+ 140us) server_negotiation.cc:407] Connection header received
0812 14:51:09.507552 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.507553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.507562 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.507623 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:09.507777 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.508492 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510049 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.510653 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510727 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.510735 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.510900 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.510937 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:51:09.511008 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.507318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:42341 (local address 127.13.146.131:35965)
0812 14:51:09.507419 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.507422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.507432 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:09.507508 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.507509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.507520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.507584 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:09.507754 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.508492 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510050 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.510639 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510726 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.510735 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.510900 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.510937 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 14:51:09.511045 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.507272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:35965 (local address 127.13.146.130:42341)
0812 14:51:09.507368 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.507380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.507430 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.507596 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.507597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.507599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:09.507669 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.507672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.508585 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.508586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.509955 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.509958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.510736 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.510823 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.510825 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.510893 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.510951 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":78}
I20260812 14:51:09.511052 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.507273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43961 (local address 127.13.146.130:35907)
0812 14:51:09.507444 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.507453 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.507507 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.507644 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.507645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.507647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:09.507708 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.507709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.508585 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.508586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.509955 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.509958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.510736 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.510737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.510818 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.510823 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.510893 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.510951 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":91,"thread_start_us":102,"threads_started":1}
I20260812 14:51:09.511397 16226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d54bfbaaa44bcb88b8c90ae9295001" candidate_uuid: "7e45c988f1664b9db6bb640f2400b492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b12020f0d1d465786fd1b35df234193" is_pre_election: true
I20260812 14:51:09.511569 16226 raft_consensus.cc:2454] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45c988f1664b9db6bb640f2400b492 in term 0.
I20260812 14:51:09.511829 15870 leader_election.cc:302] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b12020f0d1d465786fd1b35df234193, 7e45c988f1664b9db6bb640f2400b492; no voters:
I20260812 14:51:09.511818 15745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d54bfbaaa44bcb88b8c90ae9295001" candidate_uuid: "7e45c988f1664b9db6bb640f2400b492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243eb887d62f4fc385a7b5f77c4734d8" is_pre_election: true
I20260812 14:51:09.511960 15745 raft_consensus.cc:2454] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e45c988f1664b9db6bb640f2400b492 in term 0.
I20260812 14:51:09.511967 16343 raft_consensus.cc:2781] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:09.512012 16343 raft_consensus.cc:482] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:09.512042 16343 raft_consensus.cc:3037] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:09.512785 16343 raft_consensus.cc:504] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.512936 16343 leader_election.cc:288] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [CANDIDATE]: Term 1 election: Requested vote from peers 2b12020f0d1d465786fd1b35df234193 (127.13.146.131:35965), 243eb887d62f4fc385a7b5f77c4734d8 (127.13.146.129:43961)
I20260812 14:51:09.513067 16226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d54bfbaaa44bcb88b8c90ae9295001" candidate_uuid: "7e45c988f1664b9db6bb640f2400b492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b12020f0d1d465786fd1b35df234193"
I20260812 14:51:09.513082 15745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d54bfbaaa44bcb88b8c90ae9295001" candidate_uuid: "7e45c988f1664b9db6bb640f2400b492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243eb887d62f4fc385a7b5f77c4734d8"
I20260812 14:51:09.513154 16226 raft_consensus.cc:3037] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:09.513154 15745 raft_consensus.cc:3037] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:09.514091 16226 raft_consensus.cc:2454] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45c988f1664b9db6bb640f2400b492 in term 1.
I20260812 14:51:09.514094 15745 raft_consensus.cc:2454] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e45c988f1664b9db6bb640f2400b492 in term 1.
I20260812 14:51:09.514271 15870 leader_election.cc:302] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b12020f0d1d465786fd1b35df234193, 7e45c988f1664b9db6bb640f2400b492; no voters:
I20260812 14:51:09.514384 16343 raft_consensus.cc:2781] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:09.514580 16343 raft_consensus.cc:686] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 1 LEADER]: Becoming Leader. State: Replica: 7e45c988f1664b9db6bb640f2400b492, State: Running, Role: LEADER
I20260812 14:51:09.514684 16343 consensus_queue.cc:237] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } }
I20260812 14:51:09.515515 15492 catalog_manager.cc:5495] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e45c988f1664b9db6bb640f2400b492 (127.13.146.130). New cstate: current_term: 1 leader_uuid: "7e45c988f1664b9db6bb640f2400b492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e45c988f1664b9db6bb640f2400b492" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 35091 } health_report { overall_health: HEALTHY } } }
W20260812 14:51:09.517343 16078 tablet.cc:2369] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:09.528887 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.526395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.13.146.130:35091)
0812 14:51:09.526469 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.526471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.526481 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:09.526554 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.526555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.526577 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.526679 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:09.526825 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.527305 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.528078 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.528294 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.528378 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.528542 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:09.528645 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:09.528758 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.528790 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.528802 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:51:09.530676 16226 raft_consensus.cc:1261] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 1 FOLLOWER]: Refusing update from remote peer 7e45c988f1664b9db6bb640f2400b492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:09.530689 15745 raft_consensus.cc:1261] T c8d54bfbaaa44bcb88b8c90ae9295001 P 243eb887d62f4fc385a7b5f77c4734d8 [term 1 FOLLOWER]: Refusing update from remote peer 7e45c988f1664b9db6bb640f2400b492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:09.531060 16351 consensus_queue.cc:1035] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b12020f0d1d465786fd1b35df234193" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 35965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:09.531216 16343 consensus_queue.cc:1035] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243eb887d62f4fc385a7b5f77c4734d8" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:09.533595 16355 mvcc.cc:205] Tried to move back new op lower bound from 7317693519995047936 to 7317693519932755968. Current Snapshot: MvccSnapshot[applied={T|T < 7317693519995047936}]
I20260812 14:51:09.533957 16358 mvcc.cc:205] Tried to move back new op lower bound from 7317693519995047936 to 7317693519932755968. Current Snapshot: MvccSnapshot[applied={T|T < 7317693519995047936}]
I20260812 14:51:09.534724 16357 mvcc.cc:205] Tried to move back new op lower bound from 7317693519995047936 to 7317693519932755968. Current Snapshot: MvccSnapshot[applied={T|T < 7317693519995047936}]
I20260812 14:51:09.536001 15492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:09.536113 15492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:09.536866 15492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54bb1c7cef3a4fb9b73ab80347c661e4: Sending DeleteTablet for 3 replicas of tablet c8d54bfbaaa44bcb88b8c90ae9295001
I20260812 14:51:09.537202 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:09 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:58046
I20260812 14:51:09.537416 16199 tablet_service.cc:1514] Processing DeleteTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:09 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:55098
I20260812 14:51:09.537530 15710 tablet_service.cc:1514] Processing DeleteTablet for tablet c8d54bfbaaa44bcb88b8c90ae9295001 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:09 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:57798
I20260812 14:51:09.537720 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15598
I20260812 14:51:09.560148 16379 tablet_replica.cc:330] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: stopping tablet replica
I20260812 14:51:09.560322 16379 raft_consensus.cc:2229] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:51:09.560457 16379 raft_consensus.cc:2258] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:09.560792 16381 tablet_replica.cc:330] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: stopping tablet replica
I20260812 14:51:09.560940 16381 raft_consensus.cc:2229] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:09.561072 16381 raft_consensus.cc:2258] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:09.561177 16379 ts_tablet_manager.cc:1905] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:09.561619 16381 ts_tablet_manager.cc:1905] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:09.563155 16379 ts_tablet_manager.cc:1918] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:09.563323 16379 log.cc:1196] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/c8d54bfbaaa44bcb88b8c90ae9295001
I20260812 14:51:09.563679 16381 ts_tablet_manager.cc:1918] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:09.563771 16381 log.cc:1196] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/c8d54bfbaaa44bcb88b8c90ae9295001
I20260812 14:51:09.563714 16379 ts_tablet_manager.cc:1939] T c8d54bfbaaa44bcb88b8c90ae9295001 P 7e45c988f1664b9db6bb640f2400b492: Deleting consensus metadata
I20260812 14:51:09.564043 16381 ts_tablet_manager.cc:1939] T c8d54bfbaaa44bcb88b8c90ae9295001 P 2b12020f0d1d465786fd1b35df234193: Deleting consensus metadata
I20260812 14:51:09.564414 15469 catalog_manager.cc:4841] TS 7e45c988f1664b9db6bb640f2400b492 (127.13.146.130:35091): tablet c8d54bfbaaa44bcb88b8c90ae9295001 (table test-table [id=368814bb2dfb4bd6b66d620808d039a1]) successfully deleted
I20260812 14:51:09.564482 15467 catalog_manager.cc:4841] TS 2b12020f0d1d465786fd1b35df234193 (127.13.146.131:35965): tablet c8d54bfbaaa44bcb88b8c90ae9295001 (table test-table [id=368814bb2dfb4bd6b66d620808d039a1]) successfully deleted
W20260812 14:51:09.565673 15470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:43961 (error 108)
I20260812 14:51:09.566299 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.566149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43961 (local address 127.0.0.1:57808)
0812 14:51:09.566215 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.566263 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:51:09.566345 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15839
W20260812 14:51:09.566452 15470 catalog_manager.cc:4584] TS 243eb887d62f4fc385a7b5f77c4734d8 (127.13.146.129:43961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8d54bfbaaa44bcb88b8c90ae9295001: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43961: connect: Connection refused (error 111)
I20260812 14:51:09.581070 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16086
I20260812 14:51:09.593673 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 15441
2026-08-12T14:51:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2282 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T14:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:09Z Disabled control of system clock
I20260812 14:51:09.632234 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37947
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:46751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:09.720252 16389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:09.720502 16389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:09.720525 16389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:09.721731 16389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:09.721779 16389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:09.721792 16389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:09.721805 16389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:09.721817 16389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:09.723206 16389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:46751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:37947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:09.723483 16389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:09.723799 16389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:09.728981 16398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.729115 16389 server_base.cc:1034] running on GCE node
W20260812 14:51:09.728968 16399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.728968 16402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.729503 16389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:09.729817 16389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:09.730988 16389 hybrid_clock.cc:648] HybridClock initialized: now 1786546269730978 us; error 41 us; skew 500 ppm
I20260812 14:51:09.732312 16389 webserver.cc:460] Webserver started at http://127.13.146.190:39937/ using document root <none> and password file <none>
I20260812 14:51:09.732573 16389 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:09.732646 16389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:09.732810 16389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:09.733978 16389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "8e0388b4158d48e283001e82cbe339b0"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "7991295dcaef653f387b3d5865f599e5"
server_key_iv: "d2fbed203af5fefdd86f6bcce195c261"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.734361 16389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "8e0388b4158d48e283001e82cbe339b0"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "7991295dcaef653f387b3d5865f599e5"
server_key_iv: "d2fbed203af5fefdd86f6bcce195c261"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.735707 16389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:51:09.736435 16411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.736630 16389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:09.736748 16389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "8e0388b4158d48e283001e82cbe339b0"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "7991295dcaef653f387b3d5865f599e5"
server_key_iv: "d2fbed203af5fefdd86f6bcce195c261"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.736826 16389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:09.752931 16389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:09.753392 16389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:09.753561 16389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:09.757625 16389 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:37947
I20260812 14:51:09.757676 16493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:37947 every 8 connection(s)
I20260812 14:51:09.758050 16389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:09.759122 16494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:09.761714 16494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: Bootstrap starting.
I20260812 14:51:09.762311 16494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:09.762589 16494 log.cc:824] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:09.763329 16494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: No bootstrap required, opened a new log
I20260812 14:51:09.764920 16494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } }
I20260812 14:51:09.765074 16494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:09.765097 16494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0388b4158d48e283001e82cbe339b0, State: Initialized, Role: FOLLOWER
I20260812 14:51:09.765202 16494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } }
I20260812 14:51:09.765338 16494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:09.765362 16494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:09.765396 16494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:09.765878 16494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } }
I20260812 14:51:09.765965 16494 leader_election.cc:302] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e0388b4158d48e283001e82cbe339b0; no voters:
I20260812 14:51:09.766103 16494 leader_election.cc:288] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:09.766165 16498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:09.766372 16498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8e0388b4158d48e283001e82cbe339b0, State: Running, Role: LEADER
I20260812 14:51:09.766386 16494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:09.766505 16498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } }
I20260812 14:51:09.766971 16500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e0388b4158d48e283001e82cbe339b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } } }
I20260812 14:51:09.767064 16500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:09.767163 16504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0388b4158d48e283001e82cbe339b0. Latest consensus state: current_term: 1 leader_uuid: "8e0388b4158d48e283001e82cbe339b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0388b4158d48e283001e82cbe339b0" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 37947 } } }
I20260812 14:51:09.767290 16504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:09.767683 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16389
I20260812 14:51:09.767794 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:09.768045 13898 external_mini_cluster.cc:1428] Setting key 53bb0377e0c54f15125117724fdfb3cf
W20260812 14:51:09.768515 16529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:09.768728 16529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:09.768864 16530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:09.769058 16530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:09.770838 16530 catalog_manager.cc:1347] Generated new cluster ID: 9996437cf03a423b9f1920d11052e7ff
I20260812 14:51:09.770895 16530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:09.775341 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.768575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.13.146.190:37947)
0812 14:51:09.768908 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.768918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.768951 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:51:09.769882 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.769890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.769962 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.770102 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.770519 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.771901 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.773069 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.773738 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.773853 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.774615 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.774619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.774619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:09.774835 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.774875 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:09.774889 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.774950 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.775040 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.775116 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":84,"threads_started":1}
I20260812 14:51:09.785077 16530 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:09.785684 16530 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:09.790534 16530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: Generated new TSK 0
I20260812 14:51:09.790745 16530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:09.795405 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--builtin_ntp_servers=127.13.146.148:46751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:09.884879 16539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:09.885140 16539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:09.885185 16539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:09.886402 16539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:09.886469 16539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:09.886513 16539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:09.887905 16539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:46751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:09.888203 16539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:09.888521 16539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:09.895004 16545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.894999 16550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:09.895000 16556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:09.895539 16539 server_base.cc:1034] running on GCE node
I20260812 14:51:09.895759 16539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:09.896032 16539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:09.897200 16539 hybrid_clock.cc:648] HybridClock initialized: now 1786546269897192 us; error 42 us; skew 500 ppm
I20260812 14:51:09.898696 16539 webserver.cc:460] Webserver started at http://127.13.146.129:46833/ using document root <none> and password file <none>
I20260812 14:51:09.898962 16539 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:09.899019 16539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:09.899147 16539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:09.900695 16539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "44fd3e90221b49b991165f33115b0bcd"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "46e7d05857da35048c24045636fb43b0"
server_key_iv: "e2e725347fda0e51305c1bbf4f244d49"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.901139 16539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "44fd3e90221b49b991165f33115b0bcd"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "46e7d05857da35048c24045636fb43b0"
server_key_iv: "e2e725347fda0e51305c1bbf4f244d49"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.902818 16539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:09.903715 16571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.903920 16539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:09.904027 16539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "44fd3e90221b49b991165f33115b0bcd"
format_stamp: "Formatted at 2026-08-12 14:51:09 on dist-test-slave-8hhm"
server_key: "46e7d05857da35048c24045636fb43b0"
server_key_iv: "e2e725347fda0e51305c1bbf4f244d49"
server_key_version: "encryptionkey@0"
I20260812 14:51:09.904106 16539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:09.925444 16539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:09.925757 16539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:09.925863 16539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:09.926075 16539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:09.926368 16539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:09.926399 16539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.926422 16539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:09.926434 16539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:09.932737 16539 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:43735
I20260812 14:51:09.932789 16777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:43735 every 8 connection(s)
I20260812 14:51:09.933139 16539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:51:09.935112 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16539
I20260812 14:51:09.935218 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:09.935456 13898 external_mini_cluster.cc:1428] Setting key 6ccdfa727df01f2ea60e2e7c1cd1699a
I20260812 14:51:09.936537 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--builtin_ntp_servers=127.13.146.148:46751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:51:09.939337 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.934006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:33819 (local address 127.13.146.190:37947)
0812 14:51:09.934080 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:09.934082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:09.934323 (+ 241us) server_negotiation.cc:407] Connection header received
0812 14:51:09.934814 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:09.934815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:09.934828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:09.934882 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:09.935761 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.936458 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.937649 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.937984 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.938062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.939025 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:09.939027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:09.939027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:09.939032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:09.939048 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:09.939052 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:09.939150 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:09.939249 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:51:09.939276 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:51:09.939605 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:09.933985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37947 (local address 127.13.146.129:33819)
0812 14:51:09.934193 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:09.934228 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:09.934728 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:09.934935 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:09.934938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:09.934947 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:09.935637 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.935645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.936571 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.936572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.937563 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:09.937566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:09.938074 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:09.938075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:09.938248 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:09.938861 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:09.938862 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:09.938904 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:09.938934 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:09.939156 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:09.939162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:09.939165 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:09.939238 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:51:09.939345 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260812 14:51:09.940152 16785 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37947
I20260812 14:51:09.940268 16785 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:09.940460 16785 heartbeater.cc:511] Master 127.13.146.190:37947 requested a full tablet report, sending...
I20260812 14:51:09.941063 16439 ts_manager.cc:194] Registered new tserver with Master: 44fd3e90221b49b991165f33115b0bcd (127.13.146.129:43735)
I20260812 14:51:09.942108 16439 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.129:33819
W20260812 14:51:10.022670 16794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:10.022878 16794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:10.022900 16794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:10.024160 16794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:10.024205 16794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:10.024231 16794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:10.025628 16794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:46751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:10.025843 16794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:10.026106 16794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:51:10.031783 16794 server_base.cc:1034] running on GCE node
W20260812 14:51:10.031773 16802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:10.031713 16805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:10.031721 16801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:10.032191 16794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:10.032470 16794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:10.033634 16794 hybrid_clock.cc:648] HybridClock initialized: now 1786546270033616 us; error 35 us; skew 500 ppm
I20260812 14:51:10.034981 16794 webserver.cc:460] Webserver started at http://127.13.146.130:45403/ using document root <none> and password file <none>
I20260812 14:51:10.035262 16794 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:10.035336 16794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:10.035490 16794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:10.036582 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "e17a6765ae4f4b34a158553cfdd2edd4"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "ff90f69fa9f4948fb43a9eaca9376d08"
server_key_iv: "231e5a546c5f74cfd834d4e3243e3231"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.036973 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "e17a6765ae4f4b34a158553cfdd2edd4"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "ff90f69fa9f4948fb43a9eaca9376d08"
server_key_iv: "231e5a546c5f74cfd834d4e3243e3231"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.038416 16794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:10.039244 16819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.039520 16794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:10.039628 16794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "e17a6765ae4f4b34a158553cfdd2edd4"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "ff90f69fa9f4948fb43a9eaca9376d08"
server_key_iv: "231e5a546c5f74cfd834d4e3243e3231"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.039687 16794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:10.050108 16794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:10.050393 16794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:10.050498 16794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:10.050690 16794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:10.051039 16794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:10.051079 16794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.051124 16794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:10.051151 16794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.056831 16794 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:33677
I20260812 14:51:10.056913 17032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:33677 every 8 connection(s)
I20260812 14:51:10.057317 16794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:51:10.062780 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.058110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:54853 (local address 127.13.146.190:37947)
0812 14:51:10.058200 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.058202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.058415 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:51:10.058919 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.058921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.058932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.058986 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:10.059688 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.060234 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.061206 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.061514 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.061591 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.062448 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:10.062450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:10.062450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:10.062455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:10.062471 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:10.062476 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:10.062530 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.062681 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.062709 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:51:10.062968 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.058070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37947 (local address 127.13.146.130:54853)
0812 14:51:10.058289 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.058323 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.058816 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.059047 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.059051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.059060 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:10.059584 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.059590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.060325 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.060325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.061108 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.061110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.061601 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.061602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.061721 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.062292 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:10.062292 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:10.062323 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:10.062348 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:10.062560 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:10.062568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:10.062570 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.062640 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.062733 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
I20260812 14:51:10.063542 17033 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37947
I20260812 14:51:10.063611 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 16794
I20260812 14:51:10.063651 17033 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:10.063730 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:10.063874 17033 heartbeater.cc:511] Master 127.13.146.190:37947 requested a full tablet report, sending...
I20260812 14:51:10.064005 13898 external_mini_cluster.cc:1428] Setting key d5badcb583debea59e10b486831d4722
I20260812 14:51:10.064545 16436 ts_manager.cc:194] Registered new tserver with Master: e17a6765ae4f4b34a158553cfdd2edd4 (127.13.146.130:33677)
I20260812 14:51:10.065032 16436 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.130:54853
I20260812 14:51:10.065099 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--builtin_ntp_servers=127.13.146.148:46751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:10.156248 17038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:10.156472 17038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:10.156497 17038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:10.157783 17038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:10.157828 17038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:10.157855 17038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:10.159250 17038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:46751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:10.159519 17038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:10.159838 17038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:10.165957 17048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:10.165941 17050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:10.165954 17052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:10.165992 17038 server_base.cc:1034] running on GCE node
I20260812 14:51:10.166482 17038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:10.166842 17038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:10.168030 17038 hybrid_clock.cc:648] HybridClock initialized: now 1786546270168013 us; error 46 us; skew 500 ppm
I20260812 14:51:10.169584 17038 webserver.cc:460] Webserver started at http://127.13.146.131:35489/ using document root <none> and password file <none>
I20260812 14:51:10.169863 17038 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:10.169938 17038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:10.170101 17038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:10.171262 17038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "f9512173ff39465cafa374cd46e650ed"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "77654eae3558bea0f5c56909790b2b61"
server_key_iv: "004a29bdcd766f7cd606edc3bdf9daa6"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.171655 17038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "f9512173ff39465cafa374cd46e650ed"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "77654eae3558bea0f5c56909790b2b61"
server_key_iv: "004a29bdcd766f7cd606edc3bdf9daa6"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.173086 17038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:10.173943 17071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.174156 17038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:10.174270 17038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "f9512173ff39465cafa374cd46e650ed"
format_stamp: "Formatted at 2026-08-12 14:51:10 on dist-test-slave-8hhm"
server_key: "77654eae3558bea0f5c56909790b2b61"
server_key_iv: "004a29bdcd766f7cd606edc3bdf9daa6"
server_key_version: "encryptionkey@0"
I20260812 14:51:10.174351 17038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:10.195003 17038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:10.195363 17038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:10.195523 17038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:10.195782 17038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:10.196192 17038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:10.196262 17038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.196308 17038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:10.196339 17038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:10.202379 17038 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:42741
I20260812 14:51:10.202425 17284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:42741 every 8 connection(s)
I20260812 14:51:10.202797 17038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:51:10.208405 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.203648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:49585 (local address 127.13.146.190:37947)
0812 14:51:10.203723 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.203725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.203979 (+ 254us) server_negotiation.cc:407] Connection header received
0812 14:51:10.204488 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.204490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.204501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.204563 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:10.205293 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.205798 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.206827 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.207015 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.207071 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.208172 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:10.208173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:10.208173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:10.208179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:10.208193 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:10.208197 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:10.208240 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.208313 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.208341 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:51:10.208680 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.203641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:37947 (local address 127.13.146.131:49585)
0812 14:51:10.203824 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.203881 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.204394 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.204628 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.204632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.204641 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:10.205182 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.205187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.205900 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.205901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.206744 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.206747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.207141 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.207143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.207295 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.208006 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:10.208007 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:10.208040 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:10.208072 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:10.208243 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:10.208246 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:10.208249 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.208301 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.208386 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260812 14:51:10.209304 17285 heartbeater.cc:347] Connected to a master server at 127.13.146.190:37947
I20260812 14:51:10.209406 17285 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:10.209643 17285 heartbeater.cc:511] Master 127.13.146.190:37947 requested a full tablet report, sending...
I20260812 14:51:10.210083 16436 ts_manager.cc:194] Registered new tserver with Master: f9512173ff39465cafa374cd46e650ed (127.13.146.131:42741)
I20260812 14:51:10.210489 16436 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.131:49585
I20260812 14:51:10.211637 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17038
I20260812 14:51:10.211774 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:10.212005 13898 external_mini_cluster.cc:1428] Setting key 5d4f64841f72948adfef43235321014b
I20260812 14:51:10.212991 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:10.217387 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.214454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.13.146.190:37947)
0812 14:51:10.214536 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.214538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.214667 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:51:10.214692 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.214693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.214705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.214758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:10.214924 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.215432 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.216145 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.216341 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.216397 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.217040 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:10.217041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:10.217041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:10.217045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:10.217059 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:10.217063 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:10.217143 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.217269 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.217281 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 14:51:10.219128 16436 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:10.220708 16436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:10.227286 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.222957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43735 (local address 127.0.0.1:40262)
0812 14:51:10.223194 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.223235 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.223305 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.223616 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.223619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.223621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:10.223689 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.223691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225103 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.225104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.226163 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.226167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.226984 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.226985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.227065 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.227079 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.227152 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.227207 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":95,"thread_start_us":110,"threads_started":1}
I20260812 14:51:10.227291 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.223253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.13.146.129:43735)
0812 14:51:10.223453 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.223457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.223465 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:10.223496 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.223498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.223509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.223611 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:10.223825 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225001 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.226262 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.226898 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.226977 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.227062 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.227165 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.227207 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
I20260812 14:51:10.227870 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.222974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:42741 (local address 127.0.0.1:53060)
0812 14:51:10.223707 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.223716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.223766 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.223962 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.223963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.223965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:10.224021 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.224023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225275 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.225276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.226510 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.226513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.227642 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.227643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.227708 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.227712 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.227762 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.227803 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":38,"thread_start_us":95,"threads_started":1}
I20260812 14:51:10.228199 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.223229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.13.146.131:42741)
0812 14:51:10.223399 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.223405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.223826 (+ 421us) server_negotiation.cc:407] Connection header received
0812 14:51:10.223870 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.223871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.223883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.223965 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:10.224108 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225139 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.226607 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.227317 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.227506 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.227606 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.228051 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.228096 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260812 14:51:10.228366 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.222957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:33677 (local address 127.0.0.1:34248)
0812 14:51:10.223429 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.223439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.223495 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.224328 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.224329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.224330 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:10.224358 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.224359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225316 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.225317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.226728 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.226730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.228027 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.228028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.228093 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.228097 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.228147 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.228187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":31,"thread_start_us":96,"threads_started":1}
I20260812 14:51:10.228689 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.223071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.13.146.130:33677)
0812 14:51:10.224093 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.224097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.224107 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:10.224143 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.224144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.224158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.224251 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:10.224430 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.225180 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.226815 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.227721 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.227812 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.227897 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.228544 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.228588 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":108,"threads_started":1}
I20260812 14:51:10.228914 17153 tablet_service.cc:1467] Processing CreateTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 (DEFAULT_TABLE table=test-table [id=c07c9bd8da3c4621af29c8b9be2f0b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:10.229360 17153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d142f873b2049cca0dbc28e97cb2209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:10.229439 16920 tablet_service.cc:1467] Processing CreateTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 (DEFAULT_TABLE table=test-table [id=c07c9bd8da3c4621af29c8b9be2f0b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:10.229647 16653 tablet_service.cc:1467] Processing CreateTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 (DEFAULT_TABLE table=test-table [id=c07c9bd8da3c4621af29c8b9be2f0b94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:10.229827 16920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d142f873b2049cca0dbc28e97cb2209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:10.229991 16653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d142f873b2049cca0dbc28e97cb2209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:10.232698 17305 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Bootstrap starting.
I20260812 14:51:10.232962 17310 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Bootstrap starting.
I20260812 14:51:10.233448 17311 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: Bootstrap starting.
I20260812 14:51:10.233619 17310 tablet_bootstrap.cc:654] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:10.233722 17305 tablet_bootstrap.cc:654] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:10.234005 17310 log.cc:824] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:10.234086 17311 tablet_bootstrap.cc:654] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:10.234061 17305 log.cc:824] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:10.234436 17311 log.cc:824] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:10.234802 17310 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: No bootstrap required, opened a new log
I20260812 14:51:10.234860 17310 ts_tablet_manager.cc:1397] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:51:10.235278 17305 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: No bootstrap required, opened a new log
I20260812 14:51:10.235351 17305 ts_tablet_manager.cc:1397] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 14:51:10.235494 17311 tablet_bootstrap.cc:492] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: No bootstrap required, opened a new log
I20260812 14:51:10.235589 17311 ts_tablet_manager.cc:1397] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:51:10.237269 17305 raft_consensus.cc:348] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.237448 17305 raft_consensus.cc:374] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:10.237493 17305 raft_consensus.cc:729] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17a6765ae4f4b34a158553cfdd2edd4, State: Initialized, Role: FOLLOWER
I20260812 14:51:10.237629 17305 consensus_queue.cc:260] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.237746 17310 raft_consensus.cc:348] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.237933 17305 ts_tablet_manager.cc:1428] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:51:10.237985 17033 heartbeater.cc:503] Master 127.13.146.190:37947 was elected leader, sending a full tablet report...
I20260812 14:51:10.238157 17311 raft_consensus.cc:348] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.238250 17310 raft_consensus.cc:374] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:10.238307 17311 raft_consensus.cc:374] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:10.238305 17310 raft_consensus.cc:729] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9512173ff39465cafa374cd46e650ed, State: Initialized, Role: FOLLOWER
I20260812 14:51:10.238341 17311 raft_consensus.cc:729] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fd3e90221b49b991165f33115b0bcd, State: Initialized, Role: FOLLOWER
I20260812 14:51:10.238461 17311 consensus_queue.cc:260] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.238458 17310 consensus_queue.cc:260] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.238718 17311 ts_tablet_manager.cc:1428] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:51:10.238718 17310 ts_tablet_manager.cc:1428] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 14:51:10.238775 17285 heartbeater.cc:503] Master 127.13.146.190:37947 was elected leader, sending a full tablet report...
I20260812 14:51:10.238831 16785 heartbeater.cc:503] Master 127.13.146.190:37947 was elected leader, sending a full tablet report...
W20260812 14:51:10.307931 17034 tablet.cc:2369] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:10.322156 17324 raft_consensus.cc:482] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:10.322302 17324 raft_consensus.cc:504] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.322770 17324 leader_election.cc:288] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17a6765ae4f4b34a158553cfdd2edd4 (127.13.146.130:33677), f9512173ff39465cafa374cd46e650ed (127.13.146.131:42741)
I20260812 14:51:10.326912 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.322957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:33677 (local address 127.13.146.129:59173)
0812 14:51:10.323015 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.323027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.323070 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.323281 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.323282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.323283 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:10.323374 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.323376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.324359 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.324360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.325708 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.325712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.326613 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.326614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.326702 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.326706 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.326766 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.326819 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":54}
I20260812 14:51:10.326933 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.323017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:59173 (local address 127.13.146.130:33677)
0812 14:51:10.323118 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.323120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.323128 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:10.323185 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.323185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.323193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.323258 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:10.323468 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.324258 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.325825 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.326497 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.326603 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.326612 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.326809 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.326859 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 14:51:10.326933 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.323070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:37483 (local address 127.13.146.131:42741)
0812 14:51:10.323120 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.323122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.323288 (+ 166us) server_negotiation.cc:407] Connection header received
0812 14:51:10.323394 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.323395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.323404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.323464 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:10.323593 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.324309 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.325825 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.326497 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.326603 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.326612 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.326809 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.326859 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:51:10.326918 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.323026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:42741 (local address 127.13.146.129:37483)
0812 14:51:10.323213 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.323221 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:10.323304 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:10.323468 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:10.323469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:10.323471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:10.323527 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.323529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.324382 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.324383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.325708 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:10.325712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.326614 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.326615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:10.326705 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.326707 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:51:10.326766 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:51:10.326818 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":49,"thread_start_us":118,"threads_started":1}
I20260812 14:51:10.327342 16949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d142f873b2049cca0dbc28e97cb2209" candidate_uuid: "44fd3e90221b49b991165f33115b0bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" is_pre_election: true
I20260812 14:51:10.327370 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d142f873b2049cca0dbc28e97cb2209" candidate_uuid: "44fd3e90221b49b991165f33115b0bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9512173ff39465cafa374cd46e650ed" is_pre_election: true
I20260812 14:51:10.327523 16949 raft_consensus.cc:2454] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fd3e90221b49b991165f33115b0bcd in term 0.
I20260812 14:51:10.327533 17195 raft_consensus.cc:2454] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fd3e90221b49b991165f33115b0bcd in term 0.
I20260812 14:51:10.327761 16579 leader_election.cc:302] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44fd3e90221b49b991165f33115b0bcd, e17a6765ae4f4b34a158553cfdd2edd4; no voters:
I20260812 14:51:10.327929 17324 raft_consensus.cc:2781] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:10.328011 17324 raft_consensus.cc:482] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:10.328048 17324 raft_consensus.cc:3037] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:10.328779 17324 raft_consensus.cc:504] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.328945 17324 leader_election.cc:288] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [CANDIDATE]: Term 1 election: Requested vote from peers e17a6765ae4f4b34a158553cfdd2edd4 (127.13.146.130:33677), f9512173ff39465cafa374cd46e650ed (127.13.146.131:42741)
I20260812 14:51:10.329089 16949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d142f873b2049cca0dbc28e97cb2209" candidate_uuid: "44fd3e90221b49b991165f33115b0bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17a6765ae4f4b34a158553cfdd2edd4"
I20260812 14:51:10.329146 17195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d142f873b2049cca0dbc28e97cb2209" candidate_uuid: "44fd3e90221b49b991165f33115b0bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9512173ff39465cafa374cd46e650ed"
I20260812 14:51:10.329197 16949 raft_consensus.cc:3037] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:10.329288 17195 raft_consensus.cc:3037] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:10.329977 16949 raft_consensus.cc:2454] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fd3e90221b49b991165f33115b0bcd in term 1.
I20260812 14:51:10.330049 17195 raft_consensus.cc:2454] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fd3e90221b49b991165f33115b0bcd in term 1.
I20260812 14:51:10.330235 16572 leader_election.cc:302] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44fd3e90221b49b991165f33115b0bcd, f9512173ff39465cafa374cd46e650ed; no voters:
I20260812 14:51:10.330331 17324 raft_consensus.cc:2781] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:10.330528 17324 raft_consensus.cc:686] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [term 1 LEADER]: Becoming Leader. State: Replica: 44fd3e90221b49b991165f33115b0bcd, State: Running, Role: LEADER
I20260812 14:51:10.330631 17324 consensus_queue.cc:237] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } }
I20260812 14:51:10.331498 16436 catalog_manager.cc:5495] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd reported cstate change: term changed from 0 to 1, leader changed from <none> to 44fd3e90221b49b991165f33115b0bcd (127.13.146.129). New cstate: current_term: 1 leader_uuid: "44fd3e90221b49b991165f33115b0bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd3e90221b49b991165f33115b0bcd" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 43735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 } health_report { overall_health: UNKNOWN } } }
I20260812 14:51:10.343814 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.341262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.13.146.129:43735)
0812 14:51:10.341328 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:10.341330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:10.341341 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:51:10.341413 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:10.341414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:10.341441 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:10.341515 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:10.341677 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.342239 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.343023 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:10.343234 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:10.343297 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:10.343480 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:10.343572 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:10.343650 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:10.343725 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:51:10.343736 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:51:10.345806 16949 raft_consensus.cc:1261] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 1 FOLLOWER]: Refusing update from remote peer 44fd3e90221b49b991165f33115b0bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:10.345805 17195 raft_consensus.cc:1261] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 1 FOLLOWER]: Refusing update from remote peer 44fd3e90221b49b991165f33115b0bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:10.346231 17327 consensus_queue.cc:1035] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9512173ff39465cafa374cd46e650ed" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 42741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:10.346396 17324 consensus_queue.cc:1035] T 7d142f873b2049cca0dbc28e97cb2209 P 44fd3e90221b49b991165f33115b0bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17a6765ae4f4b34a158553cfdd2edd4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:10.348855 17335 mvcc.cc:205] Tried to move back new op lower bound from 7317693523333709824 to 7317693523274899456. Current Snapshot: MvccSnapshot[applied={T|T < 7317693523333709824}]
I20260812 14:51:10.351351 16439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:10.351467 16439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:10.352262 16439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e0388b4158d48e283001e82cbe339b0: Sending DeleteTablet for 3 replicas of tablet 7d142f873b2049cca0dbc28e97cb2209
I20260812 14:51:10.352659 17153 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:10 UTC) from {username='slave'} at 127.0.0.1:53060
I20260812 14:51:10.352689 16653 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:10 UTC) from {username='slave'} at 127.0.0.1:40262
I20260812 14:51:10.352757 16920 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d142f873b2049cca0dbc28e97cb2209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:10 UTC) from {username='slave'} at 127.0.0.1:34248
I20260812 14:51:10.352979 17355 tablet_replica.cc:330] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: stopping tablet replica
I20260812 14:51:10.353070 17355 raft_consensus.cc:2229] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:10.353169 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16539
I20260812 14:51:10.358801 17355 raft_consensus.cc:2258] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:10.380612 17353 tablet_replica.cc:330] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: stopping tablet replica
I20260812 14:51:10.380795 17353 raft_consensus.cc:2229] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:10.380928 17353 raft_consensus.cc:2258] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:10.382318 17353 ts_tablet_manager.cc:1905] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:10.382388 17355 ts_tablet_manager.cc:1905] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:10.384178 17353 ts_tablet_manager.cc:1918] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:10.384171 17355 ts_tablet_manager.cc:1918] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:10.384261 17355 log.cc:1196] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/7d142f873b2049cca0dbc28e97cb2209
I20260812 14:51:10.384261 17353 log.cc:1196] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/7d142f873b2049cca0dbc28e97cb2209
I20260812 14:51:10.384538 17353 ts_tablet_manager.cc:1939] T 7d142f873b2049cca0dbc28e97cb2209 P f9512173ff39465cafa374cd46e650ed: Deleting consensus metadata
I20260812 14:51:10.384538 17355 ts_tablet_manager.cc:1939] T 7d142f873b2049cca0dbc28e97cb2209 P e17a6765ae4f4b34a158553cfdd2edd4: Deleting consensus metadata
I20260812 14:51:10.385013 16418 catalog_manager.cc:4841] TS e17a6765ae4f4b34a158553cfdd2edd4 (127.13.146.130:33677): tablet 7d142f873b2049cca0dbc28e97cb2209 (table test-table [id=c07c9bd8da3c4621af29c8b9be2f0b94]) successfully deleted
I20260812 14:51:10.385051 16413 catalog_manager.cc:4841] TS f9512173ff39465cafa374cd46e650ed (127.13.146.131:42741): tablet 7d142f873b2049cca0dbc28e97cb2209 (table test-table [id=c07c9bd8da3c4621af29c8b9be2f0b94]) successfully deleted
W20260812 14:51:10.386996 16414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:43735 (error 108)
I20260812 14:51:10.387244 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16794
I20260812 14:51:10.390059 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:10.389916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:43735 (local address 127.0.0.1:40278)
0812 14:51:10.389976 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:10.390026 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 14:51:10.390234 16414 catalog_manager.cc:4584] TS 44fd3e90221b49b991165f33115b0bcd (127.13.146.129:43735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d142f873b2049cca0dbc28e97cb2209: Network error: Client connection negotiation failed: client connection to 127.13.146.129:43735: connect: Connection refused (error 111)
I20260812 14:51:10.399531 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17038
I20260812 14:51:10.419464 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 16389
2026-08-12T14:51:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (825 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:10 dist-test-slave-8hhm krb5kdc[17359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:10 dist-test-slave-8hhm krb5kdc[17359](info): set up 2 sockets
Aug 12 14:51:10 dist-test-slave-8hhm krb5kdc[17359](info): commencing operation
krb5kdc: starting...
W20260812 14:51:11.777276 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:11 dist-test-slave-8hhm krb5kdc[17359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:11.908814 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34695
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:11.993108 17386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:11.993410 17386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:11.993453 17386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:11.994639 17386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:11.994714 17386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:11.994735 17386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:11.994755 17386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:11.994787 17386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:11.996300 17386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:11.996598 17386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:11.996833 17386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:12.002326 17395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.002345 17398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.002462 17386 server_base.cc:1034] running on GCE node
W20260812 14:51:12.002328 17394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.002959 17386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:12.003381 17386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:12.004592 17386 hybrid_clock.cc:648] HybridClock initialized: now 1786546272004579 us; error 48 us; skew 500 ppm
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:12.011929 17386 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:12.012473 17386 webserver.cc:460] Webserver started at http://127.13.146.190:37043/ using document root <none> and password file <none>
I20260812 14:51:12.012722 17386 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:12.012797 17386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:12.012956 17386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:12.014171 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "2acb086bd3cf4558a85f58cc020ff3c3"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "a8e5e7e6a45e8f60481ac3d054afd4f4"
server_key_iv: "c09f0f3f3d35e1dfbd93289712840fce"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.014585 17386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "2acb086bd3cf4558a85f58cc020ff3c3"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "a8e5e7e6a45e8f60481ac3d054afd4f4"
server_key_iv: "c09f0f3f3d35e1dfbd93289712840fce"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.016062 17386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:12.016827 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.017006 17386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:12.017163 17386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "2acb086bd3cf4558a85f58cc020ff3c3"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "a8e5e7e6a45e8f60481ac3d054afd4f4"
server_key_iv: "c09f0f3f3d35e1dfbd93289712840fce"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.017271 17386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:12.035652 17386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:12.038254 17386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:12.038434 17386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:12.042958 17386 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34695
I20260812 14:51:12.043004 17514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34695 every 8 connection(s)
I20260812 14:51:12.043414 17386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:12.044324 17515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:12.044831 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17386
I20260812 14:51:12.044972 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:12.045192 13898 external_mini_cluster.cc:1428] Setting key 82cfcdcc8e74a54a6230e9fa7e85fede
I20260812 14:51:12.047624 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: Bootstrap starting.
I20260812 14:51:12.048265 17515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:12.048602 17515 log.cc:824] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:12.049435 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: No bootstrap required, opened a new log
I20260812 14:51:12.051303 17515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } }
I20260812 14:51:12.051477 17515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:12.051515 17515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acb086bd3cf4558a85f58cc020ff3c3, State: Initialized, Role: FOLLOWER
I20260812 14:51:12.051666 17515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } }
I20260812 14:51:12.051765 17515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:12.051798 17515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:12.051848 17515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:12.052548 17515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } }
I20260812 14:51:12.052645 17515 leader_election.cc:302] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acb086bd3cf4558a85f58cc020ff3c3; no voters:
I20260812 14:51:12.052834 17515 leader_election.cc:288] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:12.052894 17520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:12.053030 17520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [term 1 LEADER]: Becoming Leader. State: Replica: 2acb086bd3cf4558a85f58cc020ff3c3, State: Running, Role: LEADER
I20260812 14:51:12.053124 17520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } }
I20260812 14:51:12.053123 17515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:12.053471 17524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acb086bd3cf4558a85f58cc020ff3c3. Latest consensus state: current_term: 1 leader_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } } }
I20260812 14:51:12.053560 17524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:12.053541 17526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acb086bd3cf4558a85f58cc020ff3c3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34695 } } }
I20260812 14:51:12.054190 17526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:12.053943 17533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:12.055104 17533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:12.057308 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.045823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.13.146.190:34695)
0812 14:51:12.046011 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.046020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.046058 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:51:12.046465 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.046472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.046542 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.046659 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:12.047544 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.048614 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.049518 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.049942 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.050087 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.052791 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:12.052795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:12.052796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:12.052807 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:12.055550 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.056067 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.056071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.056071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.056196 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.056405 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.056407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.056407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.056663 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:12.056847 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.056933 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.057076 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260812 14:51:12.057761 17533 catalog_manager.cc:1347] Generated new cluster ID: 8e711ef898ab4b339784b06389f8e0a8
I20260812 14:51:12.057825 17533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:12.075748 17533 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:12.076510 17533 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:12.081836 17533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: Generated new TSK 0
I20260812 14:51:12.082046 17533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.146.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
Entry for principal kudu/127.13.146.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab.
I20260812 14:51:12.112491 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:12.199504 17572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:12.199777 17572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:12.199822 17572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:12.201007 17572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:12.201083 17572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:12.201139 17572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:12.202737 17572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.129.keytab
--principal=kudu/127.13.146.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:12.203039 17572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:12.203317 17572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:12.208936 17585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.209012 17572 server_base.cc:1034] running on GCE node
W20260812 14:51:12.208966 17583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.208937 17582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.209486 17572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:12.209807 17572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:12.210958 17572 hybrid_clock.cc:648] HybridClock initialized: now 1786546272210946 us; error 30 us; skew 500 ppm
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:12.213893 17572 init.cc:377] Logged in from keytab as kudu/127.13.146.129@KRBTEST.COM (short username kudu)
I20260812 14:51:12.214453 17572 webserver.cc:460] Webserver started at http://127.13.146.129:33767/ using document root <none> and password file <none>
I20260812 14:51:12.214699 17572 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:12.214747 17572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:12.214915 17572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:12.216048 17572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "807446ffa4e34493bf2ad467147ce816"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "cfea80949b0fd05a4a8ea4ec6e0ab9a3"
server_key_iv: "bb9dc7c47a4cdd3d81dab87003b43125"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.216445 17572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "807446ffa4e34493bf2ad467147ce816"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "cfea80949b0fd05a4a8ea4ec6e0ab9a3"
server_key_iv: "bb9dc7c47a4cdd3d81dab87003b43125"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.217962 17572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:12.218798 17597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.218994 17572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:12.219074 17572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "807446ffa4e34493bf2ad467147ce816"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "cfea80949b0fd05a4a8ea4ec6e0ab9a3"
server_key_iv: "bb9dc7c47a4cdd3d81dab87003b43125"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.219156 17572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:12.229272 17572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:12.231712 17572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:12.231845 17572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:12.232044 17572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:12.232414 17572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:12.232452 17572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.232496 17572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:12.232522 17572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.238643 17572 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:36427
I20260812 14:51:12.238703 17808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:36427 every 8 connection(s)
I20260812 14:51:12.239022 17572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.129@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:12.248353 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.239772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:38477 (local address 127.13.146.190:34695)
0812 14:51:12.239869 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.239872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.240020 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:51:12.240581 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.240582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.240598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.240652 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:12.241755 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.242253 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.243281 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.243481 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.243534 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.245690 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:12.245691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:12.245691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:12.245698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:12.246984 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.247468 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.247469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.247469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.247526 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.247794 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.247795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.247795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.247901 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:12.247953 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.248209 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.248283 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 14:51:12.248665 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17572
I20260812 14:51:12.248765 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:12.248950 13898 external_mini_cluster.cc:1428] Setting key e5c0aabeb125fa7060a48ec644209389
I20260812 14:51:12.249353 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.239717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34695 (local address 127.13.146.129:38477)
0812 14:51:12.239907 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.239948 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.240487 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.240723 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.240727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.241448 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:12.241656 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.241662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.242348 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.242349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.243197 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.243199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.243556 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.243557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.243658 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.244124 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:12.244133 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:12.245586 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:12.247151 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.247159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.247160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.247392 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.247611 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.247612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.247612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.247729 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.247960 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:12.247965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:12.248088 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.248178 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.248396 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260812 14:51:12.249933 17809 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34695
I20260812 14:51:12.250069 17809 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:12.250280 17809 heartbeater.cc:511] Master 127.13.146.190:34695 requested a full tablet report, sending...
I20260812 14:51:12.250844 17456 ts_manager.cc:194] Registered new tserver with Master: 807446ffa4e34493bf2ad467147ce816 (127.13.146.129:36427)
I20260812 14:51:12.251511 17456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.129@KRBTEST.COM'} at 127.13.146.129:38477
WARNING: no policy specified for kudu/127.13.146.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
Entry for principal kudu/127.13.146.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab.
I20260812 14:51:12.277097 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:12.362172 17820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:12.362389 17820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:12.362414 17820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:12.363608 17820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:12.363655 17820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:12.363682 17820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:12.365187 17820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.130.keytab
--principal=kudu/127.13.146.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:12.365459 17820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:12.365717 17820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:12.371140 17834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.371140 17830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.371196 17832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.371157 17820 server_base.cc:1034] running on GCE node
I20260812 14:51:12.371529 17820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:12.371814 17820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:12.372973 17820 hybrid_clock.cc:648] HybridClock initialized: now 1786546272372918 us; error 80 us; skew 500 ppm
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:12.375905 17820 init.cc:377] Logged in from keytab as kudu/127.13.146.130@KRBTEST.COM (short username kudu)
I20260812 14:51:12.376405 17820 webserver.cc:460] Webserver started at http://127.13.146.130:32955/ using document root <none> and password file <none>
I20260812 14:51:12.376643 17820 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:12.376693 17820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:12.376823 17820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:12.377998 17820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "24d6a2e9109441439e56934af2f490c4"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "396d21a8b0c928271a667122d8259e0f"
server_key_iv: "e56a423ef0bedfb86c604c2755802f43"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.378384 17820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "24d6a2e9109441439e56934af2f490c4"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "396d21a8b0c928271a667122d8259e0f"
server_key_iv: "e56a423ef0bedfb86c604c2755802f43"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.379812 17820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:51:12.380671 17850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.380897 17820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:12.381003 17820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "24d6a2e9109441439e56934af2f490c4"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "396d21a8b0c928271a667122d8259e0f"
server_key_iv: "e56a423ef0bedfb86c604c2755802f43"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.381107 17820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:12.388780 17820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:12.391119 17820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:12.391301 17820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:12.391543 17820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:12.392145 17820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:12.392215 17820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.392262 17820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:12.392314 17820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.398257 17820 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:34771
I20260812 14:51:12.398293 18085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:34771 every 8 connection(s)
I20260812 14:51:12.398627 17820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:51:12.402694 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 17820
I20260812 14:51:12.402854 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:12.403101 13898 external_mini_cluster.cc:1428] Setting key 13470b829ae3020d304c5b08f20fb425
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.130@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:12.409396 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.399494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:51461 (local address 127.13.146.190:34695)
0812 14:51:12.399558 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.399561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.399804 (+ 243us) server_negotiation.cc:407] Connection header received
0812 14:51:12.400326 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.400327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.400337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.400392 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:12.401443 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.401921 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.402982 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.403281 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.403352 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.405990 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:12.405992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:12.405992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:12.406002 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:12.407739 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.408158 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.408160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.408160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.408230 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.408482 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.408483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.408483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.408662 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:12.408745 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.409159 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.409276 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:51:12.409503 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.399485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34695 (local address 127.13.146.130:51461)
0812 14:51:12.399681 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.399718 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.400234 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.400449 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.400452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.401051 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:12.401336 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.401345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.402008 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.402009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.402843 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.402846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.403379 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.403380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.403516 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.404114 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:12.404123 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:12.405894 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:12.407830 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.407837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.407839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.408080 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.408300 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.408301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.408301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.408410 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.408775 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:12.408781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:12.408985 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.409098 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.409301 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260812 14:51:12.410007 18087 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34695
I20260812 14:51:12.410126 18087 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:12.410338 18087 heartbeater.cc:511] Master 127.13.146.190:34695 requested a full tablet report, sending...
I20260812 14:51:12.410789 17456 ts_manager.cc:194] Registered new tserver with Master: 24d6a2e9109441439e56934af2f490c4 (127.13.146.130:34771)
I20260812 14:51:12.411361 17456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.130@KRBTEST.COM'} at 127.13.146.130:51461
WARNING: no policy specified for kudu/127.13.146.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
Entry for principal kudu/127.13.146.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab.
I20260812 14:51:12.432785 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:12.518314 18096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:12.518539 18096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:12.518564 18096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:12.519687 18096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:12.519726 18096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:12.519753 18096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:12.521509 18096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.131.keytab
--principal=kudu/127.13.146.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:34695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:12.521751 18096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:12.521948 18096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:12.527388 18110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.527395 18112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.527405 18106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.527741 18096 server_base.cc:1034] running on GCE node
I20260812 14:51:12.527985 18096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:12.528270 18096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:12.529426 18096 hybrid_clock.cc:648] HybridClock initialized: now 1786546272529411 us; error 28 us; skew 500 ppm
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:12.532109 18096 init.cc:377] Logged in from keytab as kudu/127.13.146.131@KRBTEST.COM (short username kudu)
I20260812 14:51:12.532557 18096 webserver.cc:460] Webserver started at http://127.13.146.131:46709/ using document root <none> and password file <none>
I20260812 14:51:12.532801 18096 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:12.532871 18096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:12.533030 18096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:12.534209 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "19488e95fbbd4ba0813ccd82a91b78f8"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "91ccaa5723eb3d9fefc556c690ccbbea"
server_key_iv: "22c8641c973820c47ed7a5054c001486"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.534612 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "19488e95fbbd4ba0813ccd82a91b78f8"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "91ccaa5723eb3d9fefc556c690ccbbea"
server_key_iv: "22c8641c973820c47ed7a5054c001486"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.536011 18096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:12.536736 18123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.536916 18096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:12.537031 18096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "19488e95fbbd4ba0813ccd82a91b78f8"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "91ccaa5723eb3d9fefc556c690ccbbea"
server_key_iv: "22c8641c973820c47ed7a5054c001486"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.537113 18096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:12.550719 18096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:12.552785 18096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:12.552945 18096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:12.553128 18096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:12.553498 18096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:12.553544 18096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.553575 18096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:12.553622 18096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.559566 18096 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:41847
I20260812 14:51:12.559638 18336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:41847 every 8 connection(s)
I20260812 14:51:12.559990 18096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:51:12 dist-test-slave-8hhm krb5kdc[17359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546272, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.131@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:12.568895 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18096
I20260812 14:51:12.569090 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:12.569357 13898 external_mini_cluster.cc:1428] Setting key bbe6807d09c117b5c5ef7cecbae691c0
I20260812 14:51:12.569707 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.560763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:52169 (local address 127.13.146.190:34695)
0812 14:51:12.560827 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.560829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.561035 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:51:12.561640 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.561641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.561654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.561728 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:12.562710 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.563217 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.564250 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.564459 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.564536 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.566819 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:12.566821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:12.566821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:12.566829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:12.568201 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.568568 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.568569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.568570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.568634 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.568891 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.568892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.568892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.569034 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:12.569110 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.569485 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.569588 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:51:12.569917 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.560747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:34695 (local address 127.13.146.131:52169)
0812 14:51:12.560916 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.560951 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.561530 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.561808 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.561811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.562401 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:12.562593 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.562598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.563318 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.563319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.564117 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.564119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.564543 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.564544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.564672 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.565271 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:12.565281 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:12.566725 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:12.568276 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.568282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.568283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.568500 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.568691 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:12.568692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:12.568692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:12.568814 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:12.569116 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:12.569124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:12.569322 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.569451 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.569668 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260812 14:51:12.570377 18337 heartbeater.cc:347] Connected to a master server at 127.13.146.190:34695
I20260812 14:51:12.570492 18337 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:12.570669 18337 heartbeater.cc:511] Master 127.13.146.190:34695 requested a full tablet report, sending...
I20260812 14:51:12.571125 17456 ts_manager.cc:194] Registered new tserver with Master: 19488e95fbbd4ba0813ccd82a91b78f8 (127.13.146.131:41847)
I20260812 14:51:12.571535 17456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.146.131@KRBTEST.COM'} at 127.13.146.131:52169
I20260812 14:51:12.581472 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:12.589378 17518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.582926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.13.146.190:34695)
0812 14:51:12.582989 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.582991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.583152 (+ 161us) server_negotiation.cc:407] Connection header received
0812 14:51:12.583174 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.583175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.583184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.583225 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:12.583844 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.584382 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.585203 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.585453 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.585508 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.586760 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:12.586761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:12.586761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:12.586767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:12.588205 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.588566 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.588567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.588567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.588622 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:12.588855 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:12.588856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:12.588856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:12.588996 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:12.589072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.589187 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.589314 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:51:12.591066 17456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:12.592718 17456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:12.599751 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:41847 (local address 127.0.0.1:47470)
0812 14:51:12.595734 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.595745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.595790 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.596156 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.596159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.596162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:12.596262 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.596265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597255 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.597256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.598467 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.598470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599406 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.599500 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599517 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.599594 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.599658 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":23,"thread_start_us":74,"threads_started":1}
I20260812 14:51:12.599987 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36427 (local address 127.0.0.1:46644)
0812 14:51:12.595620 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.595663 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.595734 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.596174 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.596174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.596177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:12.596390 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.596391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597694 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.597695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.598851 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.598854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599607 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.599686 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599691 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.599749 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.599793 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":30,"thread_start_us":59,"threads_started":1}
I20260812 14:51:12.600021 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.13.146.130:34771)
0812 14:51:12.595516 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.595521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.595840 (+ 319us) server_negotiation.cc:407] Connection header received
0812 14:51:12.595890 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.595892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.595910 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.596057 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:12.596343 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597545 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.598835 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599530 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599624 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599680 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.599902 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.599942 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260812 14:51:12.600117 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.13.146.131:41847)
0812 14:51:12.595953 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.595957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.595967 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:12.596011 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.596013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.596032 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.596239 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:12.596343 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597122 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.598569 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599308 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599391 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599448 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.599982 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.600032 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
I20260812 14:51:12.600194 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34771 (local address 127.0.0.1:33494)
0812 14:51:12.595627 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.595660 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.595750 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.596156 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.596159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.596165 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:12.596250 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.596253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597675 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.597676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.598736 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.598739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599713 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.599771 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599774 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.599817 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.599855 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":46,"thread_start_us":103,"threads_started":1}
I20260812 14:51:12.600817 18210 tablet_service.cc:1467] Processing CreateTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 (DEFAULT_TABLE table=test-table [id=9355185da18842b896eaab263b10b4b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:12.601016 17947 tablet_service.cc:1467] Processing CreateTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 (DEFAULT_TABLE table=test-table [id=9355185da18842b896eaab263b10b4b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:12.601194 18210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16cf3d1fd3843c2a6f87208e83f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:12.601421 17947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16cf3d1fd3843c2a6f87208e83f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:12.600337 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.595410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.13.146.129:36427)
0812 14:51:12.595905 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.595911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.595922 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:51:12.595969 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.595971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.596002 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.596086 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:12.596476 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.597583 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.598940 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.599507 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.599597 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.599650 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.599764 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.599814 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":87,"threads_started":1}
I20260812 14:51:12.603888 18373 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Bootstrap starting.
I20260812 14:51:12.604173 18371 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Bootstrap starting.
I20260812 14:51:12.604545 17679 tablet_service.cc:1467] Processing CreateTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 (DEFAULT_TABLE table=test-table [id=9355185da18842b896eaab263b10b4b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:12.604889 18373 tablet_bootstrap.cc:654] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:12.604897 18371 tablet_bootstrap.cc:654] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:12.604907 17679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d16cf3d1fd3843c2a6f87208e83f5414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:12.605533 18373 log.cc:824] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:12.605582 18371 log.cc:824] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:12.606297 18373 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: No bootstrap required, opened a new log
I20260812 14:51:12.606382 18373 ts_tablet_manager.cc:1397] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:51:12.607852 18379 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Bootstrap starting.
I20260812 14:51:12.607928 18373 raft_consensus.cc:348] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.608137 18373 raft_consensus.cc:374] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:12.608189 18373 raft_consensus.cc:729] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19488e95fbbd4ba0813ccd82a91b78f8, State: Initialized, Role: FOLLOWER
I20260812 14:51:12.608309 18373 consensus_queue.cc:260] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.608583 18379 tablet_bootstrap.cc:654] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:12.608608 18373 ts_tablet_manager.cc:1428] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:51:12.608671 18337 heartbeater.cc:503] Master 127.13.146.190:34695 was elected leader, sending a full tablet report...
I20260812 14:51:12.609195 18379 log.cc:824] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:12.610057 18371 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: No bootstrap required, opened a new log
I20260812 14:51:12.610139 18371 ts_tablet_manager.cc:1397] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260812 14:51:12.610191 18379 tablet_bootstrap.cc:492] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: No bootstrap required, opened a new log
I20260812 14:51:12.610265 18379 ts_tablet_manager.cc:1397] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:51:12.612286 18371 raft_consensus.cc:348] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.612286 18379 raft_consensus.cc:348] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.612484 18379 raft_consensus.cc:374] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:12.612494 18371 raft_consensus.cc:374] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:12.612532 18371 raft_consensus.cc:729] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d6a2e9109441439e56934af2f490c4, State: Initialized, Role: FOLLOWER
I20260812 14:51:12.612531 18379 raft_consensus.cc:729] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807446ffa4e34493bf2ad467147ce816, State: Initialized, Role: FOLLOWER
I20260812 14:51:12.612660 18379 consensus_queue.cc:260] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.612660 18371 consensus_queue.cc:260] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.612968 18371 ts_tablet_manager.cc:1428] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:51:12.612968 18379 ts_tablet_manager.cc:1428] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:51:12.613001 18087 heartbeater.cc:503] Master 127.13.146.190:34695 was elected leader, sending a full tablet report...
I20260812 14:51:12.613162 17809 heartbeater.cc:503] Master 127.13.146.190:34695 was elected leader, sending a full tablet report...
W20260812 14:51:12.649298 18090 tablet.cc:2369] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:12.733956 18380 raft_consensus.cc:482] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:12.734108 18380 raft_consensus.cc:504] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.734560 18380 leader_election.cc:288] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d6a2e9109441439e56934af2f490c4 (127.13.146.130:34771), 807446ffa4e34493bf2ad467147ce816 (127.13.146.129:36427)
I20260812 14:51:12.737838 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.734785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36427 (local address 127.13.146.131:49747)
0812 14:51:12.734832 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.734844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.734897 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.735130 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.735131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.735139 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:12.735247 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.735249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.735985 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.735986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.736872 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.736874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.737562 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.737564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.737653 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.737657 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.737722 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.737756 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":48}
I20260812 14:51:12.737857 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.734878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:49747 (local address 127.13.146.129:36427)
0812 14:51:12.734927 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.734929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.734938 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:12.734971 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.734972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.734987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.735063 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:12.735327 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.735882 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.736974 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.737485 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.737544 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.737551 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.737729 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.737770 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:51:12.737955 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.734856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:34771 (local address 127.13.146.131:42833)
0812 14:51:12.735066 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.735077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:12.735161 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:12.735327 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:12.735328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:12.735329 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:12.735402 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.735403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.736197 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.736198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.736988 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:12.736990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.737674 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.737675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:12.737768 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.737774 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:51:12.737830 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:51:12.737867 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":139,"threads_started":1}
I20260812 14:51:12.737975 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.734930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:42833 (local address 127.13.146.130:34771)
0812 14:51:12.734981 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.734983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.735180 (+ 197us) server_negotiation.cc:407] Connection header received
0812 14:51:12.735214 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.735215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.735227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.735299 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:12.735506 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.736049 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.737105 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.737580 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.737655 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.737663 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.737854 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.737898 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:51:12.738194 17720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16cf3d1fd3843c2a6f87208e83f5414" candidate_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "807446ffa4e34493bf2ad467147ce816" is_pre_election: true
I20260812 14:51:12.738341 17980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16cf3d1fd3843c2a6f87208e83f5414" candidate_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d6a2e9109441439e56934af2f490c4" is_pre_election: true
I20260812 14:51:12.738410 17720 raft_consensus.cc:2454] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19488e95fbbd4ba0813ccd82a91b78f8 in term 0.
I20260812 14:51:12.738492 17980 raft_consensus.cc:2454] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19488e95fbbd4ba0813ccd82a91b78f8 in term 0.
I20260812 14:51:12.738682 18127 leader_election.cc:302] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19488e95fbbd4ba0813ccd82a91b78f8, 807446ffa4e34493bf2ad467147ce816; no voters:
I20260812 14:51:12.738828 18380 raft_consensus.cc:2781] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:12.738885 18380 raft_consensus.cc:482] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:12.738915 18380 raft_consensus.cc:3037] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:12.739596 18380 raft_consensus.cc:504] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.739746 18380 leader_election.cc:288] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [CANDIDATE]: Term 1 election: Requested vote from peers 24d6a2e9109441439e56934af2f490c4 (127.13.146.130:34771), 807446ffa4e34493bf2ad467147ce816 (127.13.146.129:36427)
W20260812 14:51:12.739812 17815 tablet.cc:2369] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:12.739938 17980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16cf3d1fd3843c2a6f87208e83f5414" candidate_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d6a2e9109441439e56934af2f490c4"
I20260812 14:51:12.740060 17980 raft_consensus.cc:3037] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:12.740108 17720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d16cf3d1fd3843c2a6f87208e83f5414" candidate_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "807446ffa4e34493bf2ad467147ce816"
I20260812 14:51:12.740207 17720 raft_consensus.cc:3037] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:12.740922 17980 raft_consensus.cc:2454] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19488e95fbbd4ba0813ccd82a91b78f8 in term 1.
I20260812 14:51:12.741082 17720 raft_consensus.cc:2454] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19488e95fbbd4ba0813ccd82a91b78f8 in term 1.
I20260812 14:51:12.741128 18125 leader_election.cc:302] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19488e95fbbd4ba0813ccd82a91b78f8, 24d6a2e9109441439e56934af2f490c4; no voters:
I20260812 14:51:12.741248 18380 raft_consensus.cc:2781] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:12.741413 18380 raft_consensus.cc:686] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 1 LEADER]: Becoming Leader. State: Replica: 19488e95fbbd4ba0813ccd82a91b78f8, State: Running, Role: LEADER
I20260812 14:51:12.741526 18380 consensus_queue.cc:237] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } }
I20260812 14:51:12.742445 17445 catalog_manager.cc:5495] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19488e95fbbd4ba0813ccd82a91b78f8 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19488e95fbbd4ba0813ccd82a91b78f8" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 41847 } health_report { overall_health: HEALTHY } } }
I20260812 14:51:12.782133 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.779798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.13.146.131:41847)
0812 14:51:12.779868 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:12.779870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:12.779878 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:12.779916 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:12.779917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:12.779936 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:12.780021 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:12.780134 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.780617 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.781392 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:12.781579 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:12.781638 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:12.781773 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:12.781878 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:12.781939 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:12.782012 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:51:12.782037 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:51:12.784107 17980 raft_consensus.cc:1261] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 1 FOLLOWER]: Refusing update from remote peer 19488e95fbbd4ba0813ccd82a91b78f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:12.784518 17720 raft_consensus.cc:1261] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 1 FOLLOWER]: Refusing update from remote peer 19488e95fbbd4ba0813ccd82a91b78f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:12.784502 18391 consensus_queue.cc:1035] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d6a2e9109441439e56934af2f490c4" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 34771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:12.784827 18391 consensus_queue.cc:1035] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "807446ffa4e34493bf2ad467147ce816" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:12.787119 18395 mvcc.cc:205] Tried to move back new op lower bound from 7317693533320654848 to 7317693533150081024. Current Snapshot: MvccSnapshot[applied={T|T < 7317693533320654848}]
I20260812 14:51:12.787285 18396 mvcc.cc:205] Tried to move back new op lower bound from 7317693533320654848 to 7317693533150081024. Current Snapshot: MvccSnapshot[applied={T|T < 7317693533320654848}]
I20260812 14:51:12.789819 17456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:12.789925 17456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:12.790769 17456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2acb086bd3cf4558a85f58cc020ff3c3: Sending DeleteTablet for 3 replicas of tablet d16cf3d1fd3843c2a6f87208e83f5414
I20260812 14:51:12.791113 17679 tablet_service.cc:1514] Processing DeleteTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:46644
I20260812 14:51:12.791172 18210 tablet_service.cc:1514] Processing DeleteTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:47470
I20260812 14:51:12.791188 17947 tablet_service.cc:1514] Processing DeleteTablet for tablet d16cf3d1fd3843c2a6f87208e83f5414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:12 UTC) from {username='kudu', principal='kudu/127.13.146.190@KRBTEST.COM'} at 127.0.0.1:33494
I20260812 14:51:12.791393 18412 tablet_replica.cc:330] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: stopping tablet replica
I20260812 14:51:12.791494 18412 raft_consensus.cc:2229] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:12.791525 18410 tablet_replica.cc:330] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816: stopping tablet replica
I20260812 14:51:12.791608 18412 raft_consensus.cc:2258] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:12.791707 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17572
I20260812 14:51:12.791821 18410 raft_consensus.cc:2229] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:12.791985 18410 raft_consensus.cc:2258] T d16cf3d1fd3843c2a6f87208e83f5414 P 807446ffa4e34493bf2ad467147ce816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:12.792094 18412 ts_tablet_manager.cc:1905] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:12.793629 18412 ts_tablet_manager.cc:1918] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:12.793704 18412 log.cc:1196] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/d16cf3d1fd3843c2a6f87208e83f5414
I20260812 14:51:12.793936 18412 ts_tablet_manager.cc:1939] T d16cf3d1fd3843c2a6f87208e83f5414 P 24d6a2e9109441439e56934af2f490c4: Deleting consensus metadata
I20260812 14:51:12.794111 18411 tablet_replica.cc:330] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: stopping tablet replica
I20260812 14:51:12.794202 18411 raft_consensus.cc:2229] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:51:12.794304 18411 raft_consensus.cc:2258] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:12.798004 17411 catalog_manager.cc:4841] TS 24d6a2e9109441439e56934af2f490c4 (127.13.146.130:34771): tablet d16cf3d1fd3843c2a6f87208e83f5414 (table test-table [id=9355185da18842b896eaab263b10b4b8]) successfully deleted
I20260812 14:51:12.805956 18411 ts_tablet_manager.cc:1905] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:12.807686 18411 ts_tablet_manager.cc:1918] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:12.807771 18411 log.cc:1196] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/d16cf3d1fd3843c2a6f87208e83f5414
I20260812 14:51:12.808108 18411 ts_tablet_manager.cc:1939] T d16cf3d1fd3843c2a6f87208e83f5414 P 19488e95fbbd4ba0813ccd82a91b78f8: Deleting consensus metadata
I20260812 14:51:12.808575 17411 catalog_manager.cc:4841] TS 19488e95fbbd4ba0813ccd82a91b78f8 (127.13.146.131:41847): tablet d16cf3d1fd3843c2a6f87208e83f5414 (table test-table [id=9355185da18842b896eaab263b10b4b8]) successfully deleted
W20260812 14:51:12.810101 17413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:36427 (error 108)
I20260812 14:51:12.810670 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:12.810520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:36427 (local address 127.0.0.1:46654)
0812 14:51:12.810579 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:12.810621 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 14:51:12.810788 17413 catalog_manager.cc:4584] TS 807446ffa4e34493bf2ad467147ce816 (127.13.146.129:36427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d16cf3d1fd3843c2a6f87208e83f5414: Network error: Client connection negotiation failed: client connection to 127.13.146.129:36427: connect: Connection refused (error 111)
I20260812 14:51:12.811084 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17820
I20260812 14:51:12.821198 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18096
I20260812 14:51:12.839538 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 17386
2026-08-12T14:51:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2421 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T14:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:12Z Disabled control of system clock
I20260812 14:51:12.879592 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36341
--webserver_interface=127.13.146.190
--webserver_port=0
--builtin_ntp_servers=127.13.146.148:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:36341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:12.964551 18420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:12.964778 18420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:12.964800 18420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:12.965994 18420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:12.966037 18420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:12.966049 18420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:12.966061 18420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:12.966073 18420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:12.967442 18420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:36341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:12.967685 18420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:12.967978 18420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:12.973191 18433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:12.973203 18436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.973320 18420 server_base.cc:1034] running on GCE node
W20260812 14:51:12.973213 18431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:12.973774 18420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:12.974159 18420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:12.975309 18420 hybrid_clock.cc:648] HybridClock initialized: now 1786546272975288 us; error 38 us; skew 500 ppm
I20260812 14:51:12.976596 18420 webserver.cc:460] Webserver started at http://127.13.146.190:42203/ using document root <none> and password file <none>
I20260812 14:51:12.976876 18420 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:12.976924 18420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:12.977087 18420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:12.978267 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "69c83fb932e641c8a3b693bec4d2273c"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "15c59a55c3ade243cec74e6e7f368e6d"
server_key_iv: "18ad427d38c0e2e8f603b3228a819c01"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.978691 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "69c83fb932e641c8a3b693bec4d2273c"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "15c59a55c3ade243cec74e6e7f368e6d"
server_key_iv: "18ad427d38c0e2e8f603b3228a819c01"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.980062 18420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:12.980860 18454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:12.981053 18420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:12.981174 18420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "69c83fb932e641c8a3b693bec4d2273c"
format_stamp: "Formatted at 2026-08-12 14:51:12 on dist-test-slave-8hhm"
server_key: "15c59a55c3ade243cec74e6e7f368e6d"
server_key_iv: "18ad427d38c0e2e8f603b3228a819c01"
server_key_version: "encryptionkey@0"
I20260812 14:51:12.981284 18420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:13.001523 18420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:13.001891 18420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:13.002056 18420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:13.006788 18420 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:36341
I20260812 14:51:13.006840 18553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:36341 every 8 connection(s)
I20260812 14:51:13.007226 18420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:13.008260 18555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:13.010608 18555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: Bootstrap starting.
I20260812 14:51:13.011165 18555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:13.011416 18555 log.cc:824] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:13.012101 18555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: No bootstrap required, opened a new log
I20260812 14:51:13.013801 18555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } }
I20260812 14:51:13.013945 18555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:13.013968 18555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c83fb932e641c8a3b693bec4d2273c, State: Initialized, Role: FOLLOWER
I20260812 14:51:13.014117 18555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } }
I20260812 14:51:13.014204 18555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:13.014228 18555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:13.014256 18555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:13.014678 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18420
I20260812 14:51:13.014796 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:13.014940 18555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } }
I20260812 14:51:13.015030 13898 external_mini_cluster.cc:1428] Setting key 3fefb07fe987c869e4ed6444551ca447
I20260812 14:51:13.015054 18555 leader_election.cc:302] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69c83fb932e641c8a3b693bec4d2273c; no voters:
I20260812 14:51:13.015239 18555 leader_election.cc:288] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:13.015308 18559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:13.015476 18559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [term 1 LEADER]: Becoming Leader. State: Replica: 69c83fb932e641c8a3b693bec4d2273c, State: Running, Role: LEADER
I20260812 14:51:13.015556 18555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:13.015597 18559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } }
I20260812 14:51:13.015926 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69c83fb932e641c8a3b693bec4d2273c. Latest consensus state: current_term: 1 leader_uuid: "69c83fb932e641c8a3b693bec4d2273c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } } }
I20260812 14:51:13.016005 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:13.016238 18561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69c83fb932e641c8a3b693bec4d2273c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c83fb932e641c8a3b693bec4d2273c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36341 } } }
I20260812 14:51:13.016317 18561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:13.016532 18575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:13.017105 18575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:13.018761 18575 catalog_manager.cc:1347] Generated new cluster ID: c95b85cdbf474b9a942a91df07a0b2fc
I20260812 14:51:13.018821 18575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:13.021960 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.015784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.13.146.190:36341)
0812 14:51:13.016018 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.016027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.016497 (+ 470us) server_negotiation.cc:407] Connection header received
0812 14:51:13.016649 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.016657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.016729 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.017406 (+ 677us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:13.017553 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.018546 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.019657 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.020288 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.020427 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.021172 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:13.021176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:13.021176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:13.021429 (+ 253us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:13.021472 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:13.021488 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:13.021561 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.021634 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.021731 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":77,"threads_started":1}
I20260812 14:51:13.027194 18575 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:13.027712 18575 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:13.033288 18575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: Generated new TSK 0
I20260812 14:51:13.033476 18575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:13.039774 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.129:0
--local_ip_for_outbound_sockets=127.13.146.129
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--builtin_ntp_servers=127.13.146.148:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:51:13.143070 18600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:13.143285 18600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:13.143306 18600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:13.144481 18600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:13.144526 18600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:13.144554 18600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.129
I20260812 14:51:13.145982 18600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.146.129
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.129
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:13.146281 18600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:13.146618 18600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:13.152850 18615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.152927 18619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.152842 18617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:13.153275 18600 server_base.cc:1034] running on GCE node
I20260812 14:51:13.153551 18600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:13.153829 18600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:13.155061 18600 hybrid_clock.cc:648] HybridClock initialized: now 1786546273155041 us; error 34 us; skew 500 ppm
I20260812 14:51:13.156611 18600 webserver.cc:460] Webserver started at http://127.13.146.129:41593/ using document root <none> and password file <none>
I20260812 14:51:13.156900 18600 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:13.156978 18600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:13.157186 18600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:13.158542 18600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data/instance:
uuid: "1ad8fe65373b465c976544e7d31afc57"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "9819dd6518aec5704e21338cd3d3de67"
server_key_iv: "c2f49b6cf0fa1cbac6da6aa3f3d995bf"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.158954 18600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance:
uuid: "1ad8fe65373b465c976544e7d31afc57"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "9819dd6518aec5704e21338cd3d3de67"
server_key_iv: "c2f49b6cf0fa1cbac6da6aa3f3d995bf"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.160488 18600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:13.161481 18629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.161707 18600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:13.161813 18600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal
uuid: "1ad8fe65373b465c976544e7d31afc57"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "9819dd6518aec5704e21338cd3d3de67"
server_key_iv: "c2f49b6cf0fa1cbac6da6aa3f3d995bf"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.161885 18600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:13.175843 18600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:13.176121 18600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:13.176224 18600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:13.176419 18600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:13.176812 18600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:13.176895 18600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.176931 18600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:13.176944 18600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.182694 18600 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.129:35673
I20260812 14:51:13.182741 18815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.129:35673 every 8 connection(s)
I20260812 14:51:13.183094 18600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/data/info.pb
I20260812 14:51:13.186823 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18600
I20260812 14:51:13.186951 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-0/wal/instance
I20260812 14:51:13.187258 13898 external_mini_cluster.cc:1428] Setting key b233f74f3284ef5a640b19a6f9f9f44d
I20260812 14:51:13.188488 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.130:0
--local_ip_for_outbound_sockets=127.13.146.130
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--builtin_ntp_servers=127.13.146.148:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:51:13.188979 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.184031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.129:50489 (local address 127.13.146.190:36341)
0812 14:51:13.184077 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.184086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.184291 (+ 205us) server_negotiation.cc:407] Connection header received
0812 14:51:13.184776 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.184777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.184789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.184852 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:13.185610 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.186122 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.187227 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.187549 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.187630 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.188645 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:13.188646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:13.188647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:13.188652 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:13.188670 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:13.188675 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:13.188735 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.188836 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.188868 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:51:13.189100 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.183942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36341 (local address 127.13.146.129:50489)
0812 14:51:13.184146 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.184197 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.184692 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.184906 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.184909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.184916 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:13.185516 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.185524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.186235 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.186236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.187134 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.187138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.187668 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.187669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.187821 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.188475 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:13.188476 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:13.188504 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:13.188528 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:13.188748 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:13.188754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:13.188757 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.188829 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.188917 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260812 14:51:13.190558 18816 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36341
I20260812 14:51:13.190644 18816 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:13.190848 18816 heartbeater.cc:511] Master 127.13.146.190:36341 requested a full tablet report, sending...
I20260812 14:51:13.191432 18482 ts_manager.cc:194] Registered new tserver with Master: 1ad8fe65373b465c976544e7d31afc57 (127.13.146.129:35673)
I20260812 14:51:13.192404 18482 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.129:50489
W20260812 14:51:13.273912 18819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:13.274123 18819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:13.274147 18819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:13.275307 18819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:13.275347 18819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:13.275372 18819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.130
I20260812 14:51:13.276973 18819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.146.130
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.130
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:13.277160 18819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:13.277489 18819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:13.282729 18833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.282729 18827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.282806 18829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:13.282907 18819 server_base.cc:1034] running on GCE node
I20260812 14:51:13.283272 18819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:13.283543 18819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:13.284694 18819 hybrid_clock.cc:648] HybridClock initialized: now 1786546273284671 us; error 51 us; skew 500 ppm
I20260812 14:51:13.286082 18819 webserver.cc:460] Webserver started at http://127.13.146.130:43765/ using document root <none> and password file <none>
I20260812 14:51:13.286315 18819 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:13.286358 18819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:13.286450 18819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:13.287530 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data/instance:
uuid: "0a985204febd4f518914f148751497f7"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "b39cadd5959d6c7e34910835068ff20d"
server_key_iv: "5b1415b2e3f98f0bbcc71913b523abdb"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.287848 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance:
uuid: "0a985204febd4f518914f148751497f7"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "b39cadd5959d6c7e34910835068ff20d"
server_key_iv: "5b1415b2e3f98f0bbcc71913b523abdb"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.289121 18819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:13.289932 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.290112 18819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:13.290230 18819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal
uuid: "0a985204febd4f518914f148751497f7"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "b39cadd5959d6c7e34910835068ff20d"
server_key_iv: "5b1415b2e3f98f0bbcc71913b523abdb"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.290310 18819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:13.306015 18819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:13.306354 18819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:13.306524 18819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:13.306797 18819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:13.307145 18819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:13.307205 18819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.307242 18819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:13.307291 18819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.313459 18819 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.130:36145
I20260812 14:51:13.313512 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.130:36145 every 8 connection(s)
I20260812 14:51:13.313864 18819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/data/info.pb
I20260812 14:51:13.315053 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 18819
I20260812 14:51:13.315169 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal/instance
I20260812 14:51:13.315392 13898 external_mini_cluster.cc:1428] Setting key 99b687ffbfb746541ebb221f2ca5d827
I20260812 14:51:13.316545 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.146.131:0
--local_ip_for_outbound_sockets=127.13.146.131
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--builtin_ntp_servers=127.13.146.148:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:51:13.320494 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.314649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.130:55943 (local address 127.13.146.190:36341)
0812 14:51:13.314722 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.314724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.314993 (+ 269us) server_negotiation.cc:407] Connection header received
0812 14:51:13.315738 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.315739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.315753 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.315824 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:13.316987 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.317571 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.318907 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.319221 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.319305 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.320170 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:13.320172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:13.320173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:13.320178 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:13.320196 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:13.320201 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:13.320261 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.320345 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.320376 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:51:13.320593 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.314643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36341 (local address 127.13.146.130:55943)
0812 14:51:13.314824 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.314887 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.315610 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.316113 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.316118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.316128 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:13.316903 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.316911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.317668 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.317669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.318792 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.318795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.319316 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.319317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.319426 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.320018 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:13.320019 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:13.320048 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:13.320071 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:13.320268 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:13.320273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:13.320275 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.320328 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.320407 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":99,"threads_started":1}
I20260812 14:51:13.321120 19053 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36341
I20260812 14:51:13.321260 19053 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:13.321514 19053 heartbeater.cc:511] Master 127.13.146.190:36341 requested a full tablet report, sending...
I20260812 14:51:13.321985 18482 ts_manager.cc:194] Registered new tserver with Master: 0a985204febd4f518914f148751497f7 (127.13.146.130:36145)
I20260812 14:51:13.322561 18482 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.130:55943
W20260812 14:51:13.403579 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:13.403805 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:13.403829 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:13.404932 19061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:13.404965 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:13.404989 19061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.146.131
I20260812 14:51:13.406322 19061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.146.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.146.131
--webserver_port=0
--tserver_master_addrs=127.13.146.190:36341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.146.131
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:13.406526 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:13.406780 19061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:13.412138 19072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.412165 19068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:13.412138 19067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:13.412672 19061 server_base.cc:1034] running on GCE node
I20260812 14:51:13.412901 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:13.413115 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:13.414258 19061 hybrid_clock.cc:648] HybridClock initialized: now 1786546273414249 us; error 34 us; skew 500 ppm
I20260812 14:51:13.415657 19061 webserver.cc:460] Webserver started at http://127.13.146.131:42517/ using document root <none> and password file <none>
I20260812 14:51:13.415933 19061 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:13.416010 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:13.416182 19061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:13.417524 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data/instance:
uuid: "3087bca338bd40c3bc8db5086af42084"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "c60fe9c8bfadaf3d91850c9f1428407e"
server_key_iv: "0680b901066ccbed8d73dafe08abe478"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.417857 19061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance:
uuid: "3087bca338bd40c3bc8db5086af42084"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "c60fe9c8bfadaf3d91850c9f1428407e"
server_key_iv: "0680b901066ccbed8d73dafe08abe478"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.419195 19061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:51:13.419893 19084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.420087 19061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:13.420188 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal
uuid: "3087bca338bd40c3bc8db5086af42084"
format_stamp: "Formatted at 2026-08-12 14:51:13 on dist-test-slave-8hhm"
server_key: "c60fe9c8bfadaf3d91850c9f1428407e"
server_key_iv: "0680b901066ccbed8d73dafe08abe478"
server_key_version: "encryptionkey@0"
I20260812 14:51:13.420269 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:13.434486 19061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:13.434813 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:13.434970 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:13.435201 19061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:51:13.435575 19061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:51:13.435626 19061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.435652 19061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:51:13.435670 19061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:13.441529 19061 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.131:36687
I20260812 14:51:13.441581 19302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.131:36687 every 8 connection(s)
I20260812 14:51:13.441958 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/data/info.pb
I20260812 14:51:13.443503 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19061
I20260812 14:51:13.443634 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal/instance
I20260812 14:51:13.443827 13898 external_mini_cluster.cc:1428] Setting key ec25c3e295878517bbaf26b53e026a54
I20260812 14:51:13.447558 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.442721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:42677 (local address 127.13.146.190:36341)
0812 14:51:13.442794 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.442796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.443027 (+ 231us) server_negotiation.cc:407] Connection header received
0812 14:51:13.443544 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.443545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.443559 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.443627 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:13.444648 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.445324 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.446294 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.446478 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.446528 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.447320 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:13.447321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:13.447321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:13.447325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:13.447340 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:13.447343 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:13.447386 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.447473 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.447498 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:51:13.447750 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.442730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:36341 (local address 127.13.146.131:42677)
0812 14:51:13.442890 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.442925 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.443434 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.443754 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.443758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.443767 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:13.444563 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.444570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.445409 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.445410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.446217 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.446220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.446554 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.446555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.446668 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.447188 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:51:13.447188 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:13.447218 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:51:13.447240 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:13.447401 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:13.447406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:13.447409 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.447462 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.447541 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260812 14:51:13.448239 19303 heartbeater.cc:347] Connected to a master server at 127.13.146.190:36341
I20260812 14:51:13.448339 19303 heartbeater.cc:464] Registering TS with master...
I20260812 14:51:13.448477 19303 heartbeater.cc:511] Master 127.13.146.190:36341 requested a full tablet report, sending...
I20260812 14:51:13.448817 18482 ts_manager.cc:194] Registered new tserver with Master: 3087bca338bd40c3bc8db5086af42084 (127.13.146.131:36687)
I20260812 14:51:13.449149 18482 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.146.131:42677
I20260812 14:51:13.455762 13898 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:51:13.460266 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.457301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.13.146.190:36341)
0812 14:51:13.457363 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.457366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.457476 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:51:13.457559 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.457559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.457572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.457630 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:13.457821 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.458314 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.459078 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.459295 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.459372 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.460020 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:13.460021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:13.460021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:13.460026 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:13.460040 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:13.460044 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:13.460088 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.460143 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.460174 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:51:13.461946 18482 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:51:13.463508 18482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:51:13.470472 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.13.146.130:36145)
0812 14:51:13.466717 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.466722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.466734 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:51:13.466779 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.466781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.466798 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.466882 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:13.467147 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.468299 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.469417 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470025 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470126 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470224 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.470322 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.470365 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260812 14:51:13.470587 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.131:36687 (local address 127.0.0.1:38956)
0812 14:51:13.466235 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.466278 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.466344 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.466978 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.466981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.466984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:13.467054 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.467055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.468354 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.468355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.469435 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.469437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470347 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.470399 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470403 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.470434 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.470469 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":162,"threads_started":1}
I20260812 14:51:13.470834 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.13.146.131:36687)
0812 14:51:13.466828 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.466832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.466842 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:13.466880 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.466881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.466895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.466974 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:13.467132 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.468258 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.469520 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470257 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470540 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470664 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.470692 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.470734 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":105,"threads_started":1}
I20260812 14:51:13.471016 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.13.146.129:35673)
0812 14:51:13.466844 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.466849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.466860 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:51:13.466902 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.466904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.466920 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.467130 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:13.467205 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.467870 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.469306 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470111 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470520 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470764 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.470789 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.470832 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":105,"threads_started":1}
I20260812 14:51:13.471037 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:35673 (local address 127.0.0.1:60622)
0812 14:51:13.466237 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.466279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.466342 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.467018 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.467019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.467021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:13.467055 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.467055 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.467960 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.467962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.469196 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.469199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470224 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.470297 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470302 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.470335 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.470384 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260812 14:51:13.470492 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.466295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:36145 (local address 127.0.0.1:37054)
0812 14:51:13.466447 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.466457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.466503 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.466977 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.466980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.466982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:13.467059 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.467061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.468384 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.468385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.469330 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.469333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.470125 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.470126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.470214 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.470230 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.470308 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.470370 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":8,"thread_start_us":90,"threads_started":1}
I20260812 14:51:13.471692 19181 tablet_service.cc:1467] Processing CreateTablet for tablet 58ab935461674e5d942c70f8a8da452e (DEFAULT_TABLE table=test-table [id=8ce050b60e7244b999bad90da62b94b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:13.471870 18706 tablet_service.cc:1467] Processing CreateTablet for tablet 58ab935461674e5d942c70f8a8da452e (DEFAULT_TABLE table=test-table [id=8ce050b60e7244b999bad90da62b94b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:13.472050 19181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ab935461674e5d942c70f8a8da452e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:13.472059 18936 tablet_service.cc:1467] Processing CreateTablet for tablet 58ab935461674e5d942c70f8a8da452e (DEFAULT_TABLE table=test-table [id=8ce050b60e7244b999bad90da62b94b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:51:13.472272 18706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ab935461674e5d942c70f8a8da452e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:13.472357 18936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ab935461674e5d942c70f8a8da452e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:13.474783 19324 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Bootstrap starting.
I20260812 14:51:13.475458 19324 tablet_bootstrap.cc:654] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:13.475838 19324 log.cc:824] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:13.476186 19327 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Bootstrap starting.
I20260812 14:51:13.476629 19324 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: No bootstrap required, opened a new log
I20260812 14:51:13.476691 19324 ts_tablet_manager.cc:1397] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:51:13.476825 19327 tablet_bootstrap.cc:654] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:13.477125 19327 log.cc:824] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:13.477380 19326 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: Bootstrap starting.
I20260812 14:51:13.477986 19326 tablet_bootstrap.cc:654] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:13.478293 19326 log.cc:824] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:13.479130 19326 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: No bootstrap required, opened a new log
I20260812 14:51:13.479215 19326 ts_tablet_manager.cc:1397] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:51:13.479296 19324 raft_consensus.cc:348] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.479454 19324 raft_consensus.cc:374] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:13.479477 19324 raft_consensus.cc:729] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3087bca338bd40c3bc8db5086af42084, State: Initialized, Role: FOLLOWER
I20260812 14:51:13.479624 19324 consensus_queue.cc:260] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.479905 19324 ts_tablet_manager.cc:1428] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:51:13.480033 19303 heartbeater.cc:503] Master 127.13.146.190:36341 was elected leader, sending a full tablet report...
I20260812 14:51:13.481721 19326 raft_consensus.cc:348] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.481856 19326 raft_consensus.cc:374] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:13.481878 19326 raft_consensus.cc:729] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad8fe65373b465c976544e7d31afc57, State: Initialized, Role: FOLLOWER
I20260812 14:51:13.481997 19326 consensus_queue.cc:260] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.482270 19326 ts_tablet_manager.cc:1428] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:51:13.482450 18816 heartbeater.cc:503] Master 127.13.146.190:36341 was elected leader, sending a full tablet report...
I20260812 14:51:13.482750 19327 tablet_bootstrap.cc:492] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: No bootstrap required, opened a new log
I20260812 14:51:13.482829 19327 ts_tablet_manager.cc:1397] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260812 14:51:13.484861 19327 raft_consensus.cc:348] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.484992 19327 raft_consensus.cc:374] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:13.485054 19327 raft_consensus.cc:729] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a985204febd4f518914f148751497f7, State: Initialized, Role: FOLLOWER
I20260812 14:51:13.485196 19327 consensus_queue.cc:260] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.485524 19327 ts_tablet_manager.cc:1428] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 14:51:13.485683 19053 heartbeater.cc:503] Master 127.13.146.190:36341 was elected leader, sending a full tablet report...
W20260812 14:51:13.564450 19055 tablet.cc:2369] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:51:13.571458 19338 raft_consensus.cc:482] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:51:13.571587 19338 raft_consensus.cc:504] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.571923 19338 leader_election.cc:288] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a985204febd4f518914f148751497f7 (127.13.146.130:36145), 1ad8fe65373b465c976544e7d31afc57 (127.13.146.129:35673)
I20260812 14:51:13.574823 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.572208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:42881 (local address 127.13.146.129:35673)
0812 14:51:13.572280 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.572283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.572291 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:13.572329 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.572330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.572340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.572429 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:13.572607 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.573085 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574104 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.574465 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574553 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.574561 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.574716 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.574757 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:51:13.574823 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.572128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:35673 (local address 127.13.146.131:42881)
0812 14:51:13.572175 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.572187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.572232 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.572437 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.572439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.572441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:13.572518 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.572520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.573174 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.573175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.573997 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.574000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.574558 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.574637 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.574640 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.574707 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.574747 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:51:13.574908 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.572230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.146.131:33149 (local address 127.13.146.130:36145)
0812 14:51:13.572273 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.572275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.572437 (+ 162us) server_negotiation.cc:407] Connection header received
0812 14:51:13.572501 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.572502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.572511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.572569 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:51:13.572738 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.573255 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574175 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.574543 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574630 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.574637 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.574797 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.574835 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:51:13.574955 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.572128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.130:36145 (local address 127.13.146.131:33149)
0812 14:51:13.572358 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.572367 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:13.572441 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:13.572572 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:13.572573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:13.572574 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:51:13.572655 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.572656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.573323 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.573324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.574071 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:13.574073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.574632 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.574633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:13.574720 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.574724 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:51:13.574792 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:51:13.574831 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":63,"thread_start_us":122,"threads_started":1}
I20260812 14:51:13.575253 18742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ab935461674e5d942c70f8a8da452e" candidate_uuid: "3087bca338bd40c3bc8db5086af42084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad8fe65373b465c976544e7d31afc57" is_pre_election: true
I20260812 14:51:13.575282 18968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ab935461674e5d942c70f8a8da452e" candidate_uuid: "3087bca338bd40c3bc8db5086af42084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a985204febd4f518914f148751497f7" is_pre_election: true
I20260812 14:51:13.575417 18742 raft_consensus.cc:2454] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3087bca338bd40c3bc8db5086af42084 in term 0.
I20260812 14:51:13.575434 18968 raft_consensus.cc:2454] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3087bca338bd40c3bc8db5086af42084 in term 0.
I20260812 14:51:13.575657 19086 leader_election.cc:302] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a985204febd4f518914f148751497f7, 3087bca338bd40c3bc8db5086af42084; no voters:
I20260812 14:51:13.575795 19338 raft_consensus.cc:2781] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:51:13.575860 19338 raft_consensus.cc:482] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:51:13.575884 19338 raft_consensus.cc:3037] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:13.576581 19338 raft_consensus.cc:504] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.576736 19338 leader_election.cc:288] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [CANDIDATE]: Term 1 election: Requested vote from peers 0a985204febd4f518914f148751497f7 (127.13.146.130:36145), 1ad8fe65373b465c976544e7d31afc57 (127.13.146.129:35673)
I20260812 14:51:13.576890 18968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ab935461674e5d942c70f8a8da452e" candidate_uuid: "3087bca338bd40c3bc8db5086af42084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a985204febd4f518914f148751497f7"
I20260812 14:51:13.576923 18742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ab935461674e5d942c70f8a8da452e" candidate_uuid: "3087bca338bd40c3bc8db5086af42084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad8fe65373b465c976544e7d31afc57"
I20260812 14:51:13.576993 18742 raft_consensus.cc:3037] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:13.576985 18968 raft_consensus.cc:3037] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:13.577999 18968 raft_consensus.cc:2454] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3087bca338bd40c3bc8db5086af42084 in term 1.
I20260812 14:51:13.577999 18742 raft_consensus.cc:2454] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3087bca338bd40c3bc8db5086af42084 in term 1.
I20260812 14:51:13.578229 19086 leader_election.cc:302] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a985204febd4f518914f148751497f7, 3087bca338bd40c3bc8db5086af42084; no voters:
I20260812 14:51:13.578330 19338 raft_consensus.cc:2781] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:13.578500 19338 raft_consensus.cc:686] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 1 LEADER]: Becoming Leader. State: Replica: 3087bca338bd40c3bc8db5086af42084, State: Running, Role: LEADER
I20260812 14:51:13.578606 19338 consensus_queue.cc:237] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } }
I20260812 14:51:13.579507 18482 catalog_manager.cc:5495] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3087bca338bd40c3bc8db5086af42084 (127.13.146.131). New cstate: current_term: 1 leader_uuid: "3087bca338bd40c3bc8db5086af42084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3087bca338bd40c3bc8db5086af42084" member_type: VOTER last_known_addr { host: "127.13.146.131" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 } health_report { overall_health: UNKNOWN } } }
I20260812 14:51:13.589743 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.587312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.13.146.131:36687)
0812 14:51:13.587372 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:13.587374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:13.587394 (+ 20us) server_negotiation.cc:407] Connection header received
0812 14:51:13.587432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:13.587432 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:13.587447 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:13.587513 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:13.587666 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.588147 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.588889 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:13.589095 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:13.589188 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:13.589390 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:13.589485 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:13.589572 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:13.589620 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:51:13.589638 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:51:13.591553 18968 raft_consensus.cc:1261] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 1 FOLLOWER]: Refusing update from remote peer 3087bca338bd40c3bc8db5086af42084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:13.591574 18742 raft_consensus.cc:1261] T 58ab935461674e5d942c70f8a8da452e P 1ad8fe65373b465c976544e7d31afc57 [term 1 FOLLOWER]: Refusing update from remote peer 3087bca338bd40c3bc8db5086af42084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:13.591945 19348 consensus_queue.cc:1035] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad8fe65373b465c976544e7d31afc57" member_type: VOTER last_known_addr { host: "127.13.146.129" port: 35673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:13.592092 19338 consensus_queue.cc:1035] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a985204febd4f518914f148751497f7" member_type: VOTER last_known_addr { host: "127.13.146.130" port: 36145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:13.596655 19368 mvcc.cc:205] Tried to move back new op lower bound from 7317693536628011008 to 7317693536578785280. Current Snapshot: MvccSnapshot[applied={T|T < 7317693536628011008 or (T in {7317693536628011008})}]
I20260812 14:51:13.596696 18482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:13.596498 19364 mvcc.cc:205] Tried to move back new op lower bound from 7317693536628011008 to 7317693536578785280. Current Snapshot: MvccSnapshot[applied={T|T < 7317693536628011008 or (T in {7317693536628011008})}]
I20260812 14:51:13.596779 18482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:51:13.597568 18482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69c83fb932e641c8a3b693bec4d2273c: Sending DeleteTablet for 3 replicas of tablet 58ab935461674e5d942c70f8a8da452e
I20260812 14:51:13.597949 18706 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ab935461674e5d942c70f8a8da452e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:13 UTC) from {username='slave'} at 127.0.0.1:60622
I20260812 14:51:13.597959 18936 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ab935461674e5d942c70f8a8da452e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:13 UTC) from {username='slave'} at 127.0.0.1:37054
I20260812 14:51:13.598183 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18600
I20260812 14:51:13.598196 19181 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ab935461674e5d942c70f8a8da452e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:51:13 UTC) from {username='slave'} at 127.0.0.1:38956
I20260812 14:51:13.605087 19388 tablet_replica.cc:330] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: stopping tablet replica
I20260812 14:51:13.605265 19388 raft_consensus.cc:2229] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:51:13.605415 19388 raft_consensus.cc:2258] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:13.607815 19388 ts_tablet_manager.cc:1905] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:13.609306 19385 tablet_replica.cc:330] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: stopping tablet replica
I20260812 14:51:13.609468 19385 raft_consensus.cc:2229] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:51:13.609552 19388 ts_tablet_manager.cc:1918] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:13.609627 19385 raft_consensus.cc:2258] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:51:13.609642 19388 log.cc:1196] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-2/wal/wals/58ab935461674e5d942c70f8a8da452e
I20260812 14:51:13.609892 19388 ts_tablet_manager.cc:1939] T 58ab935461674e5d942c70f8a8da452e P 3087bca338bd40c3bc8db5086af42084: Deleting consensus metadata
I20260812 14:51:13.610142 19385 ts_tablet_manager.cc:1905] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:51:13.610342 18457 catalog_manager.cc:4841] TS 3087bca338bd40c3bc8db5086af42084 (127.13.146.131:36687): tablet 58ab935461674e5d942c70f8a8da452e (table test-table [id=8ce050b60e7244b999bad90da62b94b2]) successfully deleted
I20260812 14:51:13.611553 19385 ts_tablet_manager.cc:1918] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:51:13.611641 19385 log.cc:1196] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Deleting WAL directory at /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786546020513802-13898-0/minicluster-data/ts-1/wal/wals/58ab935461674e5d942c70f8a8da452e
I20260812 14:51:13.611944 19385 ts_tablet_manager.cc:1939] T 58ab935461674e5d942c70f8a8da452e P 0a985204febd4f518914f148751497f7: Deleting consensus metadata
I20260812 14:51:13.612370 18457 catalog_manager.cc:4841] TS 0a985204febd4f518914f148751497f7 (127.13.146.130:36145): tablet 58ab935461674e5d942c70f8a8da452e (table test-table [id=8ce050b60e7244b999bad90da62b94b2]) successfully deleted
W20260812 14:51:13.614096 18463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.146.129:35673 (error 108)
I20260812 14:51:13.614383 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18819
I20260812 14:51:13.614640 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:13.614490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.129:35673 (local address 127.0.0.1:60624)
0812 14:51:13.614553 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:13.614602 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.129:35673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 14:51:13.614775 18463 catalog_manager.cc:4584] TS 1ad8fe65373b465c976544e7d31afc57 (127.13.146.129:35673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58ab935461674e5d942c70f8a8da452e: Network error: Client connection negotiation failed: client connection to 127.13.146.129:35673: connect: Connection refused (error 111)
I20260812 14:51:13.636994 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19061
I20260812 14:51:13.645283 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 18420
2026-08-12T14:51:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (807 ms)
[----------] 6 tests from EncryptionPolicyTest (9826 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:13 dist-test-slave-8hhm krb5kdc[19394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:13 dist-test-slave-8hhm krb5kdc[19394](info): set up 2 sockets
Aug 12 14:51:13 dist-test-slave-8hhm krb5kdc[19394](info): commencing operation
krb5kdc: starting...
W20260812 14:51:14.971277 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:15.102531 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:15.188417 19420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:15.188673 19420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:15.188714 19420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:15.190014 19420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:15.190089 19420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:15.190114 19420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:15.190133 19420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:15.190171 19420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:15.191727 19420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:15.192049 19420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:15.192291 19420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:15.197558 19436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:15.197566 19429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:15.197788 19420 server_base.cc:1034] running on GCE node
W20260812 14:51:15.197566 19434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:15.198138 19420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:15.198526 19420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:15.199733 19420 hybrid_clock.cc:648] HybridClock initialized: now 1786546275199722 us; error 53 us; skew 500 ppm
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:15.202375 19420 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:15.202775 19420 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260812 14:51:15.202975 19420 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:15.203016 19420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:15.203167 19420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:15.204208 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "815f7f2b25194373b965a4e045d98b25"
format_stamp: "Formatted at 2026-08-12 14:51:15 on dist-test-slave-8hhm"
server_key: "b26e96227599906252376c12bd3402c2"
server_key_iv: "2aacdf00ccbf1c5bc2eced201b2fdc6c"
server_key_version: "encryptionkey@0"
I20260812 14:51:15.204514 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "815f7f2b25194373b965a4e045d98b25"
format_stamp: "Formatted at 2026-08-12 14:51:15 on dist-test-slave-8hhm"
server_key: "b26e96227599906252376c12bd3402c2"
server_key_iv: "2aacdf00ccbf1c5bc2eced201b2fdc6c"
server_key_version: "encryptionkey@0"
I20260812 14:51:15.205890 19420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:15.206568 19445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:15.206741 19420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:15.206830 19420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "815f7f2b25194373b965a4e045d98b25"
format_stamp: "Formatted at 2026-08-12 14:51:15 on dist-test-slave-8hhm"
server_key: "b26e96227599906252376c12bd3402c2"
server_key_iv: "2aacdf00ccbf1c5bc2eced201b2fdc6c"
server_key_version: "encryptionkey@0"
I20260812 14:51:15.206885 19420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:15.222129 19420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:15.224166 19420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:15.224289 19420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:15.228367 19420 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32775
I20260812 14:51:15.228420 19529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32775 every 8 connection(s)
I20260812 14:51:15.228778 19420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:15.229748 19530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:15.232071 19530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25: Bootstrap starting.
I20260812 14:51:15.232681 19530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:15.232991 19530 log.cc:824] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:15.233777 19530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25: No bootstrap required, opened a new log
I20260812 14:51:15.235406 19530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260812 14:51:15.235571 19530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:15.235617 19530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815f7f2b25194373b965a4e045d98b25, State: Initialized, Role: FOLLOWER
I20260812 14:51:15.235790 19530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260812 14:51:15.235875 19530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:15.235920 19530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:15.235972 19530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:15.236516 19530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260812 14:51:15.236637 19530 leader_election.cc:302] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 815f7f2b25194373b965a4e045d98b25; no voters:
I20260812 14:51:15.236826 19530 leader_election.cc:288] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:15.236866 19534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:15.237044 19534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [term 1 LEADER]: Becoming Leader. State: Replica: 815f7f2b25194373b965a4e045d98b25, State: Running, Role: LEADER
I20260812 14:51:15.237123 19530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:15.237197 19534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } }
I20260812 14:51:15.237638 19537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 815f7f2b25194373b965a4e045d98b25. Latest consensus state: current_term: 1 leader_uuid: "815f7f2b25194373b965a4e045d98b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } } }
I20260812 14:51:15.237739 19537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:15.237761 19535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "815f7f2b25194373b965a4e045d98b25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815f7f2b25194373b965a4e045d98b25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32775 } } }
I20260812 14:51:15.237843 19535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:15.238065 19551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:15.238364 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19420
I20260812 14:51:15.238497 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:15.238641 19551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:15.238735 13898 external_mini_cluster.cc:1428] Setting key 9844bc085fb3ba48781d4638971e28e8
I20260812 14:51:15.240986 19551 catalog_manager.cc:1347] Generated new cluster ID: 26a0a2f75e5e432588441f3f07a649a6
I20260812 14:51:15.241040 19551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19394](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:51:15.250443 19551 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:15.251096 19551 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:15.251879 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:15.239434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33623 (local address 127.0.0.1:32775)
0812 14:51:15.239608 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:15.239625 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:15.239984 (+ 359us) server_negotiation.cc:407] Connection header received
0812 14:51:15.240107 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:15.240115 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:15.240182 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:15.241148 (+ 966us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:15.241287 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:15.242719 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:15.243854 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:15.244508 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:15.244984 (+ 476us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:15.247768 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:15.247773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:15.247773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:15.247787 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:15.249908 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:15.250416 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:15.250420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:15.250421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:15.250588 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:15.250865 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:15.250866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:15.250867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:15.251170 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:15.251394 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:15.251534 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:51:15.251643 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260812 14:51:15.252923 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:15.259861 19551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 815f7f2b25194373b965a4e045d98b25: Generated new TSK 0
I20260812 14:51:15.260059 19551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:15.262254 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:15.254531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60781 (local address 127.0.0.1:32775)
0812 14:51:15.254573 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:15.254575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:15.254658 (+ 83us) server_negotiation.cc:407] Connection header received
0812 14:51:15.254707 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:15.254708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:15.254718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:15.254783 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:15.255406 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:15.256203 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:15.257246 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:15.257573 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:15.257655 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:15.259102 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:15.259104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:15.259104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:15.259111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:15.260940 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:15.261436 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:15.261437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:15.261437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:15.261492 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:15.261737 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:15.261738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:15.261738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:15.261838 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:15.261918 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:15.262084 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 14:51:15.262138 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:51:15.264536 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19420
W20260812 14:51:15.278797 19391 connection.cc:535] client connection to 127.0.0.1:32775 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32775): Connection reset by peer (error 104)
2026-08-12T14:51:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19591](info): set up 2 sockets
Aug 12 14:51:15 dist-test-slave-8hhm krb5kdc[19591](info): commencing operation
krb5kdc: starting...
W20260812 14:51:16.575515 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:16 dist-test-slave-8hhm krb5kdc[19591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:16.710251 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:16.795336 19611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:16.795600 19611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:16.795642 19611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:16.797281 19611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:16.797379 19611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:16.797405 19611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:16.797435 19611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:16.797463 19611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:16.799139 19611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:16.799479 19611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:16.799736 19611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:16.804955 19619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:16.805004 19617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:16.805049 19611 server_base.cc:1034] running on GCE node
W20260812 14:51:16.804955 19622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:16.805548 19611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:16.805822 19611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:16.806980 19611 hybrid_clock.cc:648] HybridClock initialized: now 1786546276806957 us; error 45 us; skew 500 ppm
Aug 12 14:51:16 dist-test-slave-8hhm krb5kdc[19591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:16.809741 19611 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:16.810252 19611 webserver.cc:460] Webserver started at http://127.0.0.1:36985/ using document root <none> and password file <none>
I20260812 14:51:16.810518 19611 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:16.810590 19611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:16.810741 19611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:16.811901 19611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "dd90370ab6984821872503eb9dc50649"
format_stamp: "Formatted at 2026-08-12 14:51:16 on dist-test-slave-8hhm"
server_key: "51d78720c30d7ede8d557c231da99d42"
server_key_iv: "c185b51f21b916b5be76404ef09caefa"
server_key_version: "encryptionkey@0"
I20260812 14:51:16.812311 19611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "dd90370ab6984821872503eb9dc50649"
format_stamp: "Formatted at 2026-08-12 14:51:16 on dist-test-slave-8hhm"
server_key: "51d78720c30d7ede8d557c231da99d42"
server_key_iv: "c185b51f21b916b5be76404ef09caefa"
server_key_version: "encryptionkey@0"
I20260812 14:51:16.813783 19611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:16.814723 19638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:16.814958 19611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:16.815078 19611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "dd90370ab6984821872503eb9dc50649"
format_stamp: "Formatted at 2026-08-12 14:51:16 on dist-test-slave-8hhm"
server_key: "51d78720c30d7ede8d557c231da99d42"
server_key_iv: "c185b51f21b916b5be76404ef09caefa"
server_key_version: "encryptionkey@0"
I20260812 14:51:16.815187 19611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:16.831009 19611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:16.833632 19611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:16.833806 19611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:16.838150 19611 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37335
I20260812 14:51:16.838198 19716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37335 every 8 connection(s)
I20260812 14:51:16.838583 19611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:16.839479 19722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:16.841915 19722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649: Bootstrap starting.
I20260812 14:51:16.842471 19722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:16.842734 19722 log.cc:824] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:16.843485 19722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649: No bootstrap required, opened a new log
I20260812 14:51:16.845048 19722 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260812 14:51:16.845193 19722 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:16.845280 19722 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd90370ab6984821872503eb9dc50649, State: Initialized, Role: FOLLOWER
I20260812 14:51:16.845407 19722 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260812 14:51:16.845467 19722 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:16.845487 19722 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:16.845515 19722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:16.846009 19722 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260812 14:51:16.846163 19722 leader_election.cc:302] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd90370ab6984821872503eb9dc50649; no voters:
I20260812 14:51:16.846271 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19611
I20260812 14:51:16.846350 19722 leader_election.cc:288] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:16.846401 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:16.846395 19730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:16.846589 19730 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [term 1 LEADER]: Becoming Leader. State: Replica: dd90370ab6984821872503eb9dc50649, State: Running, Role: LEADER
I20260812 14:51:16.846673 13898 external_mini_cluster.cc:1428] Setting key 7bfdad0ae92754f4a77f56093783b768
I20260812 14:51:16.846719 19722 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:16.846725 19730 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260812 14:51:16.847270 19731 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd90370ab6984821872503eb9dc50649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260812 14:51:16.847368 19731 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:16.847522 19734 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd90370ab6984821872503eb9dc50649. Latest consensus state: current_term: 1 leader_uuid: "dd90370ab6984821872503eb9dc50649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd90370ab6984821872503eb9dc50649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260812 14:51:16.847652 19734 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:16.847657 19748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:16.848310 19748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:16.849972 19748 catalog_manager.cc:1347] Generated new cluster ID: 9eb9ec9337c644e2aab598c416ffa4e9
I20260812 14:51:16.850028 19748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:16.852843 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:16.847171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36235 (local address 127.0.0.1:37335)
0812 14:51:16.847377 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:16.847386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:16.847647 (+ 261us) server_negotiation.cc:407] Connection header received
0812 14:51:16.847779 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:16.847787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:16.847855 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:16.848003 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:16.848428 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:16.849847 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:16.850736 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:16.851295 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:16.851432 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:16.852176 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:16.852179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:16.852180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:16.852364 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:16.852395 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:16.852406 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:16.852504 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:16.852586 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:51:16.852615 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260812 14:51:16.853963 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:16.856470 19748 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:16.857139 19748 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:16.860060 19748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd90370ab6984821872503eb9dc50649: Generated new TSK 0
I20260812 14:51:16.860247 19748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:16.863474 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:16.856302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40217 (local address 127.0.0.1:37335)
0812 14:51:16.856381 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:16.856382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:16.860226 (+ 3844us) server_negotiation.cc:407] Connection header received
0812 14:51:16.860316 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:16.860317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:16.860328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:16.860386 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:16.860562 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:16.861394 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:16.862121 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:16.862461 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:16.862532 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:16.863239 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:16.863240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:16.863241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:16.863245 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:16.863259 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:16.863264 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:16.863321 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:16.863384 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 14:51:16.863399 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:51:16.865490 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19611
W20260812 14:51:16.904024 19583 connection.cc:535] client connection to 127.0.0.1:37335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37335): Connection reset by peer (error 104)
2026-08-12T14:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:16 dist-test-slave-8hhm krb5kdc[19786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:16 dist-test-slave-8hhm krb5kdc[19786](info): set up 2 sockets
Aug 12 14:51:16 dist-test-slave-8hhm krb5kdc[19786](info): commencing operation
krb5kdc: starting...
W20260812 14:51:18.212399 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:18 dist-test-slave-8hhm krb5kdc[19786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:18.341655 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:18.427352 19810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:18.427567 19810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:18.427589 19810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:18.428736 19810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:18.428781 19810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:18.428794 19810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:18.428807 19810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:18.428818 19810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:18.430382 19810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:18.430650 19810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:18.430897 19810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:18.436020 19820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:18.436034 19819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:18.436065 19823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:18.436064 19810 server_base.cc:1034] running on GCE node
I20260812 14:51:18.436585 19810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:18.436904 19810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:18.438078 19810 hybrid_clock.cc:648] HybridClock initialized: now 1786546278438060 us; error 29 us; skew 500 ppm
Aug 12 14:51:18 dist-test-slave-8hhm krb5kdc[19786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546278, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:18.454547 19810 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:18.455065 19810 webserver.cc:460] Webserver started at http://127.0.0.1:39825/ using document root <none> and password file <none>
I20260812 14:51:18.455353 19810 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:18.455430 19810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:18.455613 19810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:18.456758 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "a6b5203581a44c2e85f1bdb35b455600"
format_stamp: "Formatted at 2026-08-12 14:51:18 on dist-test-slave-8hhm"
server_key: "e81973dec1b9ee3323e075bcc79ea468"
server_key_iv: "69d53f3679af60bdc7bd5dbf01c049f6"
server_key_version: "encryptionkey@0"
I20260812 14:51:18.457141 19810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "a6b5203581a44c2e85f1bdb35b455600"
format_stamp: "Formatted at 2026-08-12 14:51:18 on dist-test-slave-8hhm"
server_key: "e81973dec1b9ee3323e075bcc79ea468"
server_key_iv: "69d53f3679af60bdc7bd5dbf01c049f6"
server_key_version: "encryptionkey@0"
I20260812 14:51:18.458568 19810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:18.459339 19833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:18.459528 19810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:51:18.459663 19810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "a6b5203581a44c2e85f1bdb35b455600"
format_stamp: "Formatted at 2026-08-12 14:51:18 on dist-test-slave-8hhm"
server_key: "e81973dec1b9ee3323e075bcc79ea468"
server_key_iv: "69d53f3679af60bdc7bd5dbf01c049f6"
server_key_version: "encryptionkey@0"
I20260812 14:51:18.459745 19810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:18.463297 19810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:18.465909 19810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:18.466084 19810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:18.470485 19810 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34609
I20260812 14:51:18.470520 19928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34609 every 8 connection(s)
I20260812 14:51:18.470887 19810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:18.471984 19930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:18.474511 19930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600: Bootstrap starting.
I20260812 14:51:18.475107 19930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:18.475354 19930 log.cc:824] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:18.476092 19930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600: No bootstrap required, opened a new log
I20260812 14:51:18.477476 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 19810
I20260812 14:51:18.477633 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:18.477845 13898 external_mini_cluster.cc:1428] Setting key c23359f4eb93c41909ca5f96edb48e42
I20260812 14:51:18.477941 19930 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260812 14:51:18.478160 19930 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:18.478205 19930 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b5203581a44c2e85f1bdb35b455600, State: Initialized, Role: FOLLOWER
I20260812 14:51:18.478418 19930 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260812 14:51:18.478541 19930 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:18.478612 19930 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:18.478699 19930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:18.479971 19930 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260812 14:51:18.480150 19930 leader_election.cc:302] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6b5203581a44c2e85f1bdb35b455600; no voters:
I20260812 14:51:18.480361 19938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:18.480335 19930 leader_election.cc:288] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:18.480536 19938 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [term 1 LEADER]: Becoming Leader. State: Replica: a6b5203581a44c2e85f1bdb35b455600, State: Running, Role: LEADER
I20260812 14:51:18.480711 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:18.478457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39587 (local address 127.0.0.1:34609)
0812 14:51:18.478725 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:18.478735 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:18.478772 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:51:18.478930 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:18.478938 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:18.478972 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:18.479189 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:18.479957 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:18.479961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:18.479962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:18.479967 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:18.480016 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:18.480033 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:18.480123 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:18.480273 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 14:51:18.480323 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":169,"threads_started":1}
I20260812 14:51:18.480737 19930 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:18.480711 19938 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } }
I20260812 14:51:18.481642 19939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6b5203581a44c2e85f1bdb35b455600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } } }
I20260812 14:51:18.481736 19939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:18.482059 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:18.482118 19940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6b5203581a44c2e85f1bdb35b455600. Latest consensus state: current_term: 1 leader_uuid: "a6b5203581a44c2e85f1bdb35b455600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b5203581a44c2e85f1bdb35b455600" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34609 } } }
I20260812 14:51:18.482215 19940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:18.483443 19951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:18.484035 19951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:18.484812 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:18.483507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58541 (local address 127.0.0.1:34609)
0812 14:51:18.483570 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:18.483572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:18.483712 (+ 140us) server_negotiation.cc:407] Connection header received
0812 14:51:18.483795 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:18.483796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:18.483803 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:18.483869 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:18.484562 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:18.484563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:18.484563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:18.484567 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:18.484583 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:18.484587 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:18.484663 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:18.484730 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:51:18.484744 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:51:18.486125 19951 catalog_manager.cc:1347] Generated new cluster ID: fba400337815418594a2c870bdd2fa56
I20260812 14:51:18.486178 19951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:18.502701 19951 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:18.502908 19951 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:18.506692 19951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6b5203581a44c2e85f1bdb35b455600: Generated new TSK 0
I20260812 14:51:18.506894 19951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:18.513851 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 19810
2026-08-12T14:51:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:18 dist-test-slave-8hhm krb5kdc[19987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:18 dist-test-slave-8hhm krb5kdc[19987](info): set up 2 sockets
Aug 12 14:51:18 dist-test-slave-8hhm krb5kdc[19987](info): commencing operation
krb5kdc: starting...
W20260812 14:51:19.943167 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[19987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:20.076665 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:20.165325 20016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:20.165540 20016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:20.165563 20016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:20.166683 20016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:20.166723 20016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:20.166736 20016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:20.166749 20016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:20.166761 20016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:20.168205 20016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:20.168411 20016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:20.168592 20016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:20.174029 20022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:20.174103 20016 server_base.cc:1034] running on GCE node
W20260812 14:51:20.174127 20023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:20.174038 20025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:20.174503 20016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:20.174834 20016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:20.175992 20016 hybrid_clock.cc:648] HybridClock initialized: now 1786546280175968 us; error 39 us; skew 500 ppm
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[19987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546280, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:20.178731 20016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:20.179240 20016 webserver.cc:460] Webserver started at http://127.0.0.1:33435/ using document root <none> and password file <none>
I20260812 14:51:20.179549 20016 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:20.179629 20016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:20.179793 20016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:20.180953 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "6388a585d8fc42e78aa2f4100bdd1d35"
format_stamp: "Formatted at 2026-08-12 14:51:20 on dist-test-slave-8hhm"
server_key: "d2cbe723a7ae0d4cd195133cd2158968"
server_key_iv: "f179825e1575a1047033706d625b4c35"
server_key_version: "encryptionkey@0"
I20260812 14:51:20.181377 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "6388a585d8fc42e78aa2f4100bdd1d35"
format_stamp: "Formatted at 2026-08-12 14:51:20 on dist-test-slave-8hhm"
server_key: "d2cbe723a7ae0d4cd195133cd2158968"
server_key_iv: "f179825e1575a1047033706d625b4c35"
server_key_version: "encryptionkey@0"
I20260812 14:51:20.182782 20016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:20.183561 20038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:20.183758 20016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:20.183867 20016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "6388a585d8fc42e78aa2f4100bdd1d35"
format_stamp: "Formatted at 2026-08-12 14:51:20 on dist-test-slave-8hhm"
server_key: "d2cbe723a7ae0d4cd195133cd2158968"
server_key_iv: "f179825e1575a1047033706d625b4c35"
server_key_version: "encryptionkey@0"
I20260812 14:51:20.183941 20016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:20.207547 20016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:20.209926 20016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:20.210106 20016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:20.214418 20016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33943
I20260812 14:51:20.214460 20139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33943 every 8 connection(s)
I20260812 14:51:20.214871 20016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:20.215865 20140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:20.218753 20140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35: Bootstrap starting.
I20260812 14:51:20.219449 20140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:20.219794 20140 log.cc:824] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:20.220585 20140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35: No bootstrap required, opened a new log
I20260812 14:51:20.222450 20140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260812 14:51:20.222609 20140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:20.222635 20140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6388a585d8fc42e78aa2f4100bdd1d35, State: Initialized, Role: FOLLOWER
I20260812 14:51:20.222813 20140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260812 14:51:20.222909 20140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:20.222972 20140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:20.223066 20140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:20.223070 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20016
I20260812 14:51:20.223209 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:20.223438 13898 external_mini_cluster.cc:1428] Setting key f8e1cd098d842766fbbf3916f83fa342
I20260812 14:51:20.223881 20140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260812 14:51:20.224032 20140 leader_election.cc:302] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6388a585d8fc42e78aa2f4100bdd1d35; no voters:
I20260812 14:51:20.224225 20140 leader_election.cc:288] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:20.224256 20153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:20.224466 20153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [term 1 LEADER]: Becoming Leader. State: Replica: 6388a585d8fc42e78aa2f4100bdd1d35, State: Running, Role: LEADER
I20260812 14:51:20.224612 20140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:20.224634 20153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } }
I20260812 14:51:20.225168 20157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } } }
I20260812 14:51:20.225428 20157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:20.225353 20158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6388a585d8fc42e78aa2f4100bdd1d35. Latest consensus state: current_term: 1 leader_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6388a585d8fc42e78aa2f4100bdd1d35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33943 } } }
I20260812 14:51:20.225704 20158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:20.225893 20166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:20.226409 20166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:20.229106 20166 catalog_manager.cc:1347] Generated new cluster ID: 52194a93582c47a78cf7606e86004bc1
I20260812 14:51:20.229239 20166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[19987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:51:20.236194 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:20.224005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53623 (local address 127.0.0.1:33943)
0812 14:51:20.224245 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:20.224255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:20.224299 (+ 44us) server_negotiation.cc:407] Connection header received
0812 14:51:20.224432 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:20.224440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:20.224503 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:20.225532 (+ 1029us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:20.225653 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:20.227171 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:20.228269 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:20.228930 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:20.229095 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:20.232017 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:20.232022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:20.232022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:20.232037 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:20.234352 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:20.234763 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:20.234767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:20.234768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:20.234936 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:20.235164 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:20.235165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:20.235165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:20.235545 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:20.235728 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:20.235807 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 14:51:20.235965 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":116,"threads_started":1}
I20260812 14:51:20.237608 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:20.241266 20166 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:20.241761 20166 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:20.244813 20166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6388a585d8fc42e78aa2f4100bdd1d35: Generated new TSK 0
I20260812 14:51:20.244989 20166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:20.247136 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:20.239033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33045 (local address 127.0.0.1:33943)
0812 14:51:20.239070 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:20.239072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:20.239252 (+ 180us) server_negotiation.cc:407] Connection header received
0812 14:51:20.239281 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:20.239282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:20.239309 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:20.239377 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:20.240006 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:20.240863 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:20.242076 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:20.242422 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:20.242467 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:20.244180 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:20.244181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:20.244182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:20.244190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:20.245942 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:20.246335 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:20.246337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:20.246337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:20.246407 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:20.246608 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:20.246609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:20.246609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:20.246751 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:20.246863 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:20.246936 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:51:20.247036 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:51:20.249432 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20016
2026-08-12T14:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[20207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[20207](info): set up 2 sockets
Aug 12 14:51:20 dist-test-slave-8hhm krb5kdc[20207](info): commencing operation
krb5kdc: starting...
W20260812 14:51:21.623502 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:21 dist-test-slave-8hhm krb5kdc[20207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:21.756381 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:21.842067 20230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:21.842325 20230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:21.842368 20230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:21.843529 20230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:21.843600 20230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:21.843623 20230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:21.843644 20230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:21.843673 20230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:21.845160 20230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:21.845449 20230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:21.845682 20230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:21.850767 20239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:21.850798 20240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:21.850767 20242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:21.851032 20230 server_base.cc:1034] running on GCE node
I20260812 14:51:21.851329 20230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:21.851636 20230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:21.852813 20230 hybrid_clock.cc:648] HybridClock initialized: now 1786546281852812 us; error 55 us; skew 500 ppm
Aug 12 14:51:21 dist-test-slave-8hhm krb5kdc[20207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:21.855492 20230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:21.855978 20230 webserver.cc:460] Webserver started at http://127.0.0.1:42299/ using document root <none> and password file <none>
I20260812 14:51:21.856225 20230 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:21.856297 20230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:21.856459 20230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:21.857640 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "3ea56c7d5944479dad611b709e29c40f"
format_stamp: "Formatted at 2026-08-12 14:51:21 on dist-test-slave-8hhm"
server_key: "6771f981cadacf7939a2fa05c020254e"
server_key_iv: "ddaed9513865475bbd8aa85fd01d1cf8"
server_key_version: "encryptionkey@0"
I20260812 14:51:21.858042 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "3ea56c7d5944479dad611b709e29c40f"
format_stamp: "Formatted at 2026-08-12 14:51:21 on dist-test-slave-8hhm"
server_key: "6771f981cadacf7939a2fa05c020254e"
server_key_iv: "ddaed9513865475bbd8aa85fd01d1cf8"
server_key_version: "encryptionkey@0"
I20260812 14:51:21.859427 20230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:51:21.860193 20253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:21.860397 20230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:21.860512 20230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "3ea56c7d5944479dad611b709e29c40f"
format_stamp: "Formatted at 2026-08-12 14:51:21 on dist-test-slave-8hhm"
server_key: "6771f981cadacf7939a2fa05c020254e"
server_key_iv: "ddaed9513865475bbd8aa85fd01d1cf8"
server_key_version: "encryptionkey@0"
I20260812 14:51:21.860601 20230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:21.876271 20230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:21.878620 20230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:21.878795 20230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:21.883318 20230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34489
I20260812 14:51:21.883351 20351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34489 every 8 connection(s)
I20260812 14:51:21.883716 20230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:21.884652 20353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:21.887109 20353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f: Bootstrap starting.
I20260812 14:51:21.887738 20353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:21.888044 20353 log.cc:824] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:21.888793 20353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f: No bootstrap required, opened a new log
I20260812 14:51:21.890545 20353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260812 14:51:21.890692 20353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:21.890717 20353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea56c7d5944479dad611b709e29c40f, State: Initialized, Role: FOLLOWER
I20260812 14:51:21.890870 20353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260812 14:51:21.890935 20353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:21.890977 20353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:21.891026 20353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:21.891587 20353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260812 14:51:21.891706 20353 leader_election.cc:302] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ea56c7d5944479dad611b709e29c40f; no voters:
I20260812 14:51:21.891901 20353 leader_election.cc:288] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:21.891948 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20230
I20260812 14:51:21.891960 20356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:21.892060 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:21.892125 20356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [term 1 LEADER]: Becoming Leader. State: Replica: 3ea56c7d5944479dad611b709e29c40f, State: Running, Role: LEADER
I20260812 14:51:21.892202 20353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:21.892210 20356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } }
I20260812 14:51:21.892308 13898 external_mini_cluster.cc:1428] Setting key 4d5bd3abe0f0e5531388d02fea0a0f64
I20260812 14:51:21.892737 20358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea56c7d5944479dad611b709e29c40f. Latest consensus state: current_term: 1 leader_uuid: "3ea56c7d5944479dad611b709e29c40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260812 14:51:21.892853 20358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:21.893420 20357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ea56c7d5944479dad611b709e29c40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ea56c7d5944479dad611b709e29c40f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34489 } } }
I20260812 14:51:21.893507 20357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:21.893810 20368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:21.894665 20368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:21.896394 20368 catalog_manager.cc:1347] Generated new cluster ID: 4323b8b59a45443590ed320e55de7b0a
I20260812 14:51:21.896453 20368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:21.899215 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:21.893188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47603 (local address 127.0.0.1:34489)
0812 14:51:21.893788 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:21.893798 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:21.893835 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:51:21.893996 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:21.894007 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:21.894087 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:21.894232 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:21.894659 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:21.896072 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:21.897082 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:21.897637 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:21.897765 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:21.898537 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:21.898541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:21.898542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:21.898732 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:21.898765 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:21.898777 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:21.898855 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:21.898932 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:51:21.898999 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":113,"threads_started":1}
I20260812 14:51:21.900223 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:21.912371 20368 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:21.912552 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:21.901688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55391 (local address 127.0.0.1:34489)
0812 14:51:21.901726 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:21.901728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:21.909524 (+ 7796us) server_negotiation.cc:407] Connection header received
0812 14:51:21.909611 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:21.909612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:21.909627 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:21.909685 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:21.909853 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:21.910377 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:21.911167 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:21.911364 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:21.911446 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:21.912257 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:21.912259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:21.912259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:21.912265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:21.912284 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:21.912289 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:21.912346 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:21.912425 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 14:51:21.912451 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:51:21.913252 20368 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:21.922748 20368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ea56c7d5944479dad611b709e29c40f: Generated new TSK 0
I20260812 14:51:21.922993 20368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:21.932833 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20230
2026-08-12T14:51:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:22 dist-test-slave-8hhm krb5kdc[20411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:22 dist-test-slave-8hhm krb5kdc[20411](info): set up 2 sockets
Aug 12 14:51:22 dist-test-slave-8hhm krb5kdc[20411](info): commencing operation
krb5kdc: starting...
W20260812 14:51:23.259635 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:23 dist-test-slave-8hhm krb5kdc[20411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:23.388711 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:23.473608 20428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:23.473824 20428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:23.473847 20428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:23.475047 20428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:23.475104 20428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:23.475118 20428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:23.475131 20428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:23.475143 20428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:23.476691 20428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:23.476967 20428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:23.477281 20428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:23.483319 20439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:23.483351 20441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:23.483319 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:23.483461 20428 server_base.cc:1034] running on GCE node
I20260812 14:51:23.483806 20428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:23.484112 20428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:23.485280 20428 hybrid_clock.cc:648] HybridClock initialized: now 1786546283485255 us; error 41 us; skew 500 ppm
Aug 12 14:51:23 dist-test-slave-8hhm krb5kdc[20411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:23.492681 20428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:23.493166 20428 webserver.cc:460] Webserver started at http://127.0.0.1:45947/ using document root <none> and password file <none>
I20260812 14:51:23.493439 20428 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:23.493490 20428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:23.493590 20428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:23.494712 20428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "519509d706774883b4f8b2e5f48ee124"
format_stamp: "Formatted at 2026-08-12 14:51:23 on dist-test-slave-8hhm"
server_key: "fbf91751f09f4af431485694c08b62bf"
server_key_iv: "ce7ae2bfae5d2f0ba485f856fa401c6d"
server_key_version: "encryptionkey@0"
I20260812 14:51:23.495050 20428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "519509d706774883b4f8b2e5f48ee124"
format_stamp: "Formatted at 2026-08-12 14:51:23 on dist-test-slave-8hhm"
server_key: "fbf91751f09f4af431485694c08b62bf"
server_key_iv: "ce7ae2bfae5d2f0ba485f856fa401c6d"
server_key_version: "encryptionkey@0"
I20260812 14:51:23.496454 20428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:23.497599 20453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:23.497789 20428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:23.497895 20428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "519509d706774883b4f8b2e5f48ee124"
format_stamp: "Formatted at 2026-08-12 14:51:23 on dist-test-slave-8hhm"
server_key: "fbf91751f09f4af431485694c08b62bf"
server_key_iv: "ce7ae2bfae5d2f0ba485f856fa401c6d"
server_key_version: "encryptionkey@0"
I20260812 14:51:23.497993 20428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:23.501350 20428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:23.503347 20428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:23.503515 20428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:23.507521 20428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42399
I20260812 14:51:23.507566 20555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42399 every 8 connection(s)
I20260812 14:51:23.507920 20428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:23.508989 20558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:23.511322 20558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: Bootstrap starting.
I20260812 14:51:23.511914 20558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:23.512177 20558 log.cc:824] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:23.512866 20558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: No bootstrap required, opened a new log
I20260812 14:51:23.514586 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20428
I20260812 14:51:23.514532 20558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } }
I20260812 14:51:23.514667 20558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:23.514726 20558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519509d706774883b4f8b2e5f48ee124, State: Initialized, Role: FOLLOWER
I20260812 14:51:23.514748 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:23.514938 13898 external_mini_cluster.cc:1428] Setting key d1d33d7bdab560de1b627cbeeaa14895
I20260812 14:51:23.514911 20558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } }
I20260812 14:51:23.514991 20558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:23.515050 20558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:23.515096 20558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:23.515928 20558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } }
I20260812 14:51:23.516044 20558 leader_election.cc:302] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 519509d706774883b4f8b2e5f48ee124; no voters:
I20260812 14:51:23.516211 20558 leader_election.cc:288] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:23.516279 20566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:23.516465 20558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:23.516496 20566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [term 1 LEADER]: Becoming Leader. State: Replica: 519509d706774883b4f8b2e5f48ee124, State: Running, Role: LEADER
I20260812 14:51:23.517397 20566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } }
I20260812 14:51:23.517616 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:23.515505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55917 (local address 127.0.0.1:42399)
0812 14:51:23.515997 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:23.516007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:23.516054 (+ 47us) server_negotiation.cc:407] Connection header received
0812 14:51:23.516182 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:23.516189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:23.516223 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:23.516373 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:23.517072 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:23.517076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:23.517076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:23.517080 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:23.517129 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:23.517143 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:23.517235 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:23.517326 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 14:51:23.517373 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":375,"threads_started":1}
I20260812 14:51:23.518095 20567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "519509d706774883b4f8b2e5f48ee124" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } } }
I20260812 14:51:23.518249 20567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:23.518126 20569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 519509d706774883b4f8b2e5f48ee124. Latest consensus state: current_term: 1 leader_uuid: "519509d706774883b4f8b2e5f48ee124" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519509d706774883b4f8b2e5f48ee124" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42399 } } }
I20260812 14:51:23.518630 20569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124 [sys.catalog]: This master's current role is: LEADER
W20260812 14:51:23.518817 20586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 14:51:23.518839 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 14:51:23.518877 20586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:23.521600 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:23.520420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46057 (local address 127.0.0.1:42399)
0812 14:51:23.520467 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:23.520470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:23.520558 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:51:23.520636 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:23.520637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:23.520642 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:23.520697 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:23.521390 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:23.521392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:23.521392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:23.521396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:23.521411 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:23.521415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:23.521462 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:23.521515 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:51:23.521530 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:51:23.525377 20587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:23.525604 20587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:23.527158 20587 catalog_manager.cc:1347] Generated new cluster ID: 07047fb474a24161b147b5551f026cb9
I20260812 14:51:23.527211 20587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:23.533605 20587 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:23.533792 20587 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:23.537487 20587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 519509d706774883b4f8b2e5f48ee124: Generated new TSK 0
I20260812 14:51:23.537674 20587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:23.543269 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20428
2026-08-12T14:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:23 dist-test-slave-8hhm krb5kdc[20608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:23 dist-test-slave-8hhm krb5kdc[20608](info): set up 2 sockets
Aug 12 14:51:23 dist-test-slave-8hhm krb5kdc[20608](info): commencing operation
krb5kdc: starting...
W20260812 14:51:24.885398 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:24 dist-test-slave-8hhm krb5kdc[20608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:25.017359 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:25.103379 20631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:25.103672 20631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:25.103713 20631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:25.104918 20631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:25.105000 20631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:25.105022 20631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:25.105043 20631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:25.105072 20631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:25.106657 20631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:25.106961 20631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:25.107206 20631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:25.112267 20639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:25.112282 20653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:25.112262 20640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:25.112602 20631 server_base.cc:1034] running on GCE node
I20260812 14:51:25.112874 20631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:25.113173 20631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:25.114368 20631 hybrid_clock.cc:648] HybridClock initialized: now 1786546285114351 us; error 32 us; skew 500 ppm
Aug 12 14:51:25 dist-test-slave-8hhm krb5kdc[20608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546285, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:25.116968 20631 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:25.117494 20631 webserver.cc:460] Webserver started at http://127.0.0.1:44259/ using document root <none> and password file <none>
I20260812 14:51:25.117759 20631 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:25.117830 20631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:25.117981 20631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:25.119071 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "9fe739d94fc44c0587e7baf51a639425"
format_stamp: "Formatted at 2026-08-12 14:51:25 on dist-test-slave-8hhm"
server_key: "5868c160ce73393b62153c4e98fdfc1f"
server_key_iv: "56efb110179e37ec1f2c9da503dd3cf6"
server_key_version: "encryptionkey@0"
I20260812 14:51:25.119446 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "9fe739d94fc44c0587e7baf51a639425"
format_stamp: "Formatted at 2026-08-12 14:51:25 on dist-test-slave-8hhm"
server_key: "5868c160ce73393b62153c4e98fdfc1f"
server_key_iv: "56efb110179e37ec1f2c9da503dd3cf6"
server_key_version: "encryptionkey@0"
I20260812 14:51:25.120802 20631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:25.121673 20670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:25.121879 20631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:51:25.121990 20631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "9fe739d94fc44c0587e7baf51a639425"
format_stamp: "Formatted at 2026-08-12 14:51:25 on dist-test-slave-8hhm"
server_key: "5868c160ce73393b62153c4e98fdfc1f"
server_key_iv: "56efb110179e37ec1f2c9da503dd3cf6"
server_key_version: "encryptionkey@0"
I20260812 14:51:25.122071 20631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:25.155746 20631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:25.157905 20631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:25.158080 20631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:25.162494 20631 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32941
I20260812 14:51:25.162533 20771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32941 every 8 connection(s)
I20260812 14:51:25.162956 20631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:25.163843 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20631
I20260812 14:51:25.163880 20772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:25.163945 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:25.164180 13898 external_mini_cluster.cc:1428] Setting key 7242eb4ae4591311483f1664b2d7d635
I20260812 14:51:25.167413 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: Bootstrap starting.
I20260812 14:51:25.168104 20772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:25.168409 20772 log.cc:824] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:25.169162 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: No bootstrap required, opened a new log
Aug 12 14:51:25 dist-test-slave-8hhm krb5kdc[20608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:51:25.171404 20772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } }
I20260812 14:51:25.171574 20772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:25.171619 20772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fe739d94fc44c0587e7baf51a639425, State: Initialized, Role: FOLLOWER
I20260812 14:51:25.171813 20772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } }
I20260812 14:51:25.171918 20772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:25.171965 20772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:25.172015 20772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:25.172785 20772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } }
I20260812 14:51:25.172906 20772 leader_election.cc:302] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fe739d94fc44c0587e7baf51a639425; no voters:
I20260812 14:51:25.173087 20772 leader_election.cc:288] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:25.173138 20777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:25.173321 20777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [term 1 LEADER]: Becoming Leader. State: Replica: 9fe739d94fc44c0587e7baf51a639425, State: Running, Role: LEADER
I20260812 14:51:25.173422 20772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:25.173424 20777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } }
I20260812 14:51:25.173794 20780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fe739d94fc44c0587e7baf51a639425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } } }
I20260812 14:51:25.173902 20780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:25.173785 20777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fe739d94fc44c0587e7baf51a639425. Latest consensus state: current_term: 1 leader_uuid: "9fe739d94fc44c0587e7baf51a639425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fe739d94fc44c0587e7baf51a639425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32941 } } }
I20260812 14:51:25.174116 20777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425 [sys.catalog]: This master's current role is: LEADER
W20260812 14:51:25.175565 20807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:25.175807 20807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:25.175879 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:25.164753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:54087 (local address 127.0.0.1:32941)
0812 14:51:25.164990 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:25.164999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:25.165034 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:51:25.165595 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:25.165605 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:25.165683 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:25.165840 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:25.166437 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:25.167805 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:25.168582 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:25.169121 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:25.169296 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:25.171829 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:25.171833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:25.171833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:25.171845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:25.174073 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:25.174604 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:25.174607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:25.174608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:25.174739 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:25.174960 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:25.174961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:25.174961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:25.175296 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:25.175490 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:25.175582 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:51:25.175702 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":115,"threads_started":1}
I20260812 14:51:25.175962 20788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:25.176141 20788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:25.176927 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:25.177846 20788 catalog_manager.cc:1347] Generated new cluster ID: 14d0c0306d014a90b16e7cf3762c5c98
I20260812 14:51:25.177904 20788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:25.188153 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:25.181654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:54169 (local address 127.0.0.1:32941)
0812 14:51:25.181695 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:25.181697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:25.181888 (+ 191us) server_negotiation.cc:407] Connection header received
0812 14:51:25.181954 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:25.181955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:25.181966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:25.182010 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:25.182566 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:25.183068 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:25.183809 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:25.184039 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:25.184117 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:25.185443 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:25.185445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:25.185445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:25.185452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:25.186944 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:25.187342 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:25.187343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:25.187343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:25.187393 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:25.187622 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:25.187623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:25.187623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:25.187744 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:25.187861 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:25.187977 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 14:51:25.188042 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260812 14:51:25.210980 20788 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:25.211771 20788 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:25.218915 20788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fe739d94fc44c0587e7baf51a639425: Generated new TSK 0
I20260812 14:51:25.219139 20788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:25.224674 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20631
W20260812 14:51:25.241039 20602 connection.cc:535] client connection to 127.0.0.1:32941 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32941): Connection reset by peer (error 104)
2026-08-12T14:51:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1677 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:25 dist-test-slave-8hhm krb5kdc[20824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:25 dist-test-slave-8hhm krb5kdc[20824](info): set up 2 sockets
Aug 12 14:51:25 dist-test-slave-8hhm krb5kdc[20824](info): commencing operation
krb5kdc: starting...
W20260812 14:51:26.612020 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:26 dist-test-slave-8hhm krb5kdc[20824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:26.748232 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:26.837417 20851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:26.837726 20851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:26.837777 20851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:26.839102 20851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:26.839190 20851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:26.839221 20851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:26.839255 20851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:26.839291 20851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:26.840955 20851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:26.841297 20851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:26.841552 20851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:26.846570 20865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:26.846582 20867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:26.846805 20851 server_base.cc:1034] running on GCE node
W20260812 14:51:26.846850 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:26.847086 20851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:26.847412 20851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:26.848575 20851 hybrid_clock.cc:648] HybridClock initialized: now 1786546286848568 us; error 46 us; skew 500 ppm
Aug 12 14:51:26 dist-test-slave-8hhm krb5kdc[20824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546286, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:26.851122 20851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:26.851557 20851 webserver.cc:460] Webserver started at http://127.0.0.1:34691/ using document root <none> and password file <none>
I20260812 14:51:26.851795 20851 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:26.851864 20851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:26.852020 20851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:26.853077 20851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "c4ea535f9d164441a573aa24534eea55"
format_stamp: "Formatted at 2026-08-12 14:51:26 on dist-test-slave-8hhm"
server_key: "9bb4971789d09e9ae63cb58f8ff08837"
server_key_iv: "c3409ed418dafb8027d36ef7c834c3c8"
server_key_version: "encryptionkey@0"
I20260812 14:51:26.853523 20851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "c4ea535f9d164441a573aa24534eea55"
format_stamp: "Formatted at 2026-08-12 14:51:26 on dist-test-slave-8hhm"
server_key: "9bb4971789d09e9ae63cb58f8ff08837"
server_key_iv: "c3409ed418dafb8027d36ef7c834c3c8"
server_key_version: "encryptionkey@0"
I20260812 14:51:26.854944 20851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:26.855691 20879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:26.855899 20851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:26.856012 20851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "c4ea535f9d164441a573aa24534eea55"
format_stamp: "Formatted at 2026-08-12 14:51:26 on dist-test-slave-8hhm"
server_key: "9bb4971789d09e9ae63cb58f8ff08837"
server_key_iv: "c3409ed418dafb8027d36ef7c834c3c8"
server_key_version: "encryptionkey@0"
I20260812 14:51:26.856088 20851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:26.878372 20851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:26.880429 20851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:26.880599 20851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:26.886343 20851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44997
I20260812 14:51:26.886399 20981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44997 every 8 connection(s)
I20260812 14:51:26.886778 20851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:26.888028 20982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:26.890430 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: Bootstrap starting.
I20260812 14:51:26.890975 20982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:26.891228 20982 log.cc:824] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:26.892014 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: No bootstrap required, opened a new log
I20260812 14:51:26.893639 20982 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260812 14:51:26.893788 20982 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:26.893812 20982 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ea535f9d164441a573aa24534eea55, State: Initialized, Role: FOLLOWER
I20260812 14:51:26.893946 20982 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260812 14:51:26.894027 20982 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:26.894050 20982 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:26.894079 20982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:26.894469 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 20851
I20260812 14:51:26.894609 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:26.894559 20982 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260812 14:51:26.894717 20982 leader_election.cc:302] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4ea535f9d164441a573aa24534eea55; no voters:
I20260812 14:51:26.894857 13898 external_mini_cluster.cc:1428] Setting key b19ebd3da3fab4b0cc169fa5a5daa21d
I20260812 14:51:26.894887 20982 leader_election.cc:288] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:26.894915 20995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:26.895040 20995 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [term 1 LEADER]: Becoming Leader. State: Replica: c4ea535f9d164441a573aa24534eea55, State: Running, Role: LEADER
I20260812 14:51:26.895123 20995 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260812 14:51:26.895146 20982 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:26.896757 20995 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ea535f9d164441a573aa24534eea55. Latest consensus state: current_term: 1 leader_uuid: "c4ea535f9d164441a573aa24534eea55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } } }
I20260812 14:51:26.896973 20996 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4ea535f9d164441a573aa24534eea55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ea535f9d164441a573aa24534eea55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } } }
I20260812 14:51:26.897109 20996 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:26.897538 20995 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55 [sys.catalog]: This master's current role is: LEADER
W20260812 14:51:26.897722 21021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:26.897761 21021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:26.897841 21022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:26.898031 21022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:26.899679 21022 catalog_manager.cc:1347] Generated new cluster ID: 61d77c9641c549efa2433bf526f05a2b
I20260812 14:51:26.899739 21022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:26.901400 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:26.895778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:44061 (local address 127.0.0.1:44997)
0812 14:51:26.895959 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:26.895969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:26.896004 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:51:26.896131 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:26.896138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:26.896207 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:26.896407 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:26.896630 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:26.898107 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:26.899111 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:26.899679 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:26.899806 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:26.900642 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:26.900646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:26.900647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:26.900881 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:26.900918 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:26.900932 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:26.901009 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:26.901145 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 14:51:26.901171 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260812 14:51:26.902379 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:26.907775 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:26.903924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:60107 (local address 127.0.0.1:44997)
0812 14:51:26.903962 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:26.903964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:26.904065 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:51:26.904104 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:26.904105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:26.904115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:26.904175 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:26.904290 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:26.905415 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:26.906388 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:26.906731 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:26.906805 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:26.907515 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:26.907524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:26.907524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:26.907529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:26.907545 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:26.907549 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:26.907616 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:26.907686 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:51:26.907699 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:51:26.928380 21022 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:26.929149 21022 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:26.932507 21022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4ea535f9d164441a573aa24534eea55: Generated new TSK 0
I20260812 14:51:26.932711 21022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:26.941380 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 20851
W20260812 14:51:26.962595 20819 connection.cc:535] client connection to 127.0.0.1:44997 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44997): Connection reset by peer (error 104)
2026-08-12T14:51:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1721 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:27 dist-test-slave-8hhm krb5kdc[21043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:27 dist-test-slave-8hhm krb5kdc[21043](info): set up 2 sockets
Aug 12 14:51:27 dist-test-slave-8hhm krb5kdc[21043](info): commencing operation
krb5kdc: starting...
W20260812 14:51:28.267978 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:28 dist-test-slave-8hhm krb5kdc[21043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:28.395664 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:28.480006 21069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:28.480275 21069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:28.480315 21069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:28.481540 21069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:28.481618 21069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:28.481643 21069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:28.481666 21069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:28.481703 21069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:28.483688 21069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:28.484032 21069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:28.484298 21069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:28.489578 21085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:28.489529 21081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:28.489529 21080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:28.489817 21069 server_base.cc:1034] running on GCE node
I20260812 14:51:28.490098 21069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:28.490417 21069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:28.491593 21069 hybrid_clock.cc:648] HybridClock initialized: now 1786546288491578 us; error 30 us; skew 500 ppm
Aug 12 14:51:28 dist-test-slave-8hhm krb5kdc[21043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546288, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:28.500185 21069 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:28.500721 21069 webserver.cc:460] Webserver started at http://127.0.0.1:36909/ using document root <none> and password file <none>
I20260812 14:51:28.500989 21069 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:28.501086 21069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:28.501286 21069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:28.502432 21069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "431625c1252c4fbe9485956d1f64f183"
format_stamp: "Formatted at 2026-08-12 14:51:28 on dist-test-slave-8hhm"
server_key: "87280bed1e7b6010358f6674fd9d101a"
server_key_iv: "52b9281d0f5b930b9443d09eb1b464d7"
server_key_version: "encryptionkey@0"
I20260812 14:51:28.502825 21069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "431625c1252c4fbe9485956d1f64f183"
format_stamp: "Formatted at 2026-08-12 14:51:28 on dist-test-slave-8hhm"
server_key: "87280bed1e7b6010358f6674fd9d101a"
server_key_iv: "52b9281d0f5b930b9443d09eb1b464d7"
server_key_version: "encryptionkey@0"
I20260812 14:51:28.504210 21069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:28.504978 21100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:28.505194 21069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:28.505313 21069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "431625c1252c4fbe9485956d1f64f183"
format_stamp: "Formatted at 2026-08-12 14:51:28 on dist-test-slave-8hhm"
server_key: "87280bed1e7b6010358f6674fd9d101a"
server_key_iv: "52b9281d0f5b930b9443d09eb1b464d7"
server_key_version: "encryptionkey@0"
I20260812 14:51:28.505373 21069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:28.508657 21069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:28.511044 21069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:28.511154 21069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:28.515218 21069 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41241
I20260812 14:51:28.515271 21208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41241 every 8 connection(s)
I20260812 14:51:28.515626 21069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:28.516664 21210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:28.519037 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183: Bootstrap starting.
I20260812 14:51:28.519666 21210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:28.519970 21210 log.cc:824] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:28.520686 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183: No bootstrap required, opened a new log
I20260812 14:51:28.521555 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21069
I20260812 14:51:28.521695 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:28.521895 13898 external_mini_cluster.cc:1428] Setting key ad0221c734514a3a1fa54c5ed7b73a30
I20260812 14:51:28.523057 21210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } }
I20260812 14:51:28.523243 21210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:28.523288 21210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431625c1252c4fbe9485956d1f64f183, State: Initialized, Role: FOLLOWER
I20260812 14:51:28.523499 21210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } }
I20260812 14:51:28.523605 21210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:28.523656 21210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:28.523710 21210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:28.524549 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:28.522465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:40187 (local address 127.0.0.1:41241)
0812 14:51:28.522678 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:28.522686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:28.522712 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:51:28.523047 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:28.523056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:28.523085 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:28.523207 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:28.523989 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:28.523992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:28.523993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:28.523996 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:28.524042 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:28.524057 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:28.524126 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:28.524228 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:51:28.524297 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260812 14:51:28.524802 21210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } }
I20260812 14:51:28.524914 21210 leader_election.cc:302] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 431625c1252c4fbe9485956d1f64f183; no voters:
I20260812 14:51:28.525072 21210 leader_election.cc:288] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:28.525085 21221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:28.525300 21221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [term 1 LEADER]: Becoming Leader. State: Replica: 431625c1252c4fbe9485956d1f64f183, State: Running, Role: LEADER
I20260812 14:51:28.525393 21210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:28.525480 21221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } }
I20260812 14:51:28.525566 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:28.525923 21221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 431625c1252c4fbe9485956d1f64f183. Latest consensus state: current_term: 1 leader_uuid: "431625c1252c4fbe9485956d1f64f183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } } }
I20260812 14:51:28.526176 21221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:28.526099 21223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "431625c1252c4fbe9485956d1f64f183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431625c1252c4fbe9485956d1f64f183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41241 } } }
I20260812 14:51:28.526646 21223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:28.526824 21238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:28.527544 21238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:28.528523 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:28.527385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:36253 (local address 127.0.0.1:41241)
0812 14:51:28.527420 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:28.527422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:28.527530 (+ 108us) server_negotiation.cc:407] Connection header received
0812 14:51:28.527574 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:28.527575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:28.527580 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:28.527644 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:28.528293 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:28.528295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:28.528295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:28.528299 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:28.528316 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:28.528320 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:28.528380 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:28.528448 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 14:51:28.528461 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:51:28.529318 21238 catalog_manager.cc:1347] Generated new cluster ID: 74a2e79e4f134ee19d6117c7c8927e17
I20260812 14:51:28.529361 21238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:28.571758 21238 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:28.571990 21238 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:28.577343 21238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 431625c1252c4fbe9485956d1f64f183: Generated new TSK 0
I20260812 14:51:28.577548 21238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:28.586534 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21069
2026-08-12T14:51:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:28 dist-test-slave-8hhm krb5kdc[21267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:28 dist-test-slave-8hhm krb5kdc[21267](info): set up 2 sockets
Aug 12 14:51:28 dist-test-slave-8hhm krb5kdc[21267](info): commencing operation
krb5kdc: starting...
W20260812 14:51:29.903592 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:29 dist-test-slave-8hhm krb5kdc[21267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:30.032801 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:30.117515 21290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:30.117726 21290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:30.117749 21290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:30.118911 21290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:30.118955 21290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:30.118969 21290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:30.118983 21290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:30.118994 21290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:30.120484 21290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:30.120735 21290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:30.120997 21290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:30.126430 21300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:30.126477 21296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:30.126461 21290 server_base.cc:1034] running on GCE node
W20260812 14:51:30.126461 21304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:30.126971 21290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:30.127290 21290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:30.128472 21290 hybrid_clock.cc:648] HybridClock initialized: now 1786546290128454 us; error 37 us; skew 500 ppm
Aug 12 14:51:30 dist-test-slave-8hhm krb5kdc[21267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546290, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:30.131062 21290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:30.131567 21290 webserver.cc:460] Webserver started at http://127.0.0.1:35729/ using document root <none> and password file <none>
I20260812 14:51:30.131819 21290 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:30.131891 21290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:30.132045 21290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:30.133137 21290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "f3b18647dc4240e6af1666571f32ca64"
format_stamp: "Formatted at 2026-08-12 14:51:30 on dist-test-slave-8hhm"
server_key: "2be38bc107ac3c37f61ed7fa318d7ef2"
server_key_iv: "e8efddcf72346e6c24b6cd98b8c23628"
server_key_version: "encryptionkey@0"
I20260812 14:51:30.133586 21290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "f3b18647dc4240e6af1666571f32ca64"
format_stamp: "Formatted at 2026-08-12 14:51:30 on dist-test-slave-8hhm"
server_key: "2be38bc107ac3c37f61ed7fa318d7ef2"
server_key_iv: "e8efddcf72346e6c24b6cd98b8c23628"
server_key_version: "encryptionkey@0"
I20260812 14:51:30.134943 21290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:30.135761 21320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:30.135963 21290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:30.136080 21290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "f3b18647dc4240e6af1666571f32ca64"
format_stamp: "Formatted at 2026-08-12 14:51:30 on dist-test-slave-8hhm"
server_key: "2be38bc107ac3c37f61ed7fa318d7ef2"
server_key_iv: "e8efddcf72346e6c24b6cd98b8c23628"
server_key_version: "encryptionkey@0"
I20260812 14:51:30.136158 21290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:30.145609 21290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:30.147413 21290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:30.147573 21290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:30.151860 21290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36699
I20260812 14:51:30.151893 21416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36699 every 8 connection(s)
I20260812 14:51:30.152238 21290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:30.153134 21417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:30.155602 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64: Bootstrap starting.
I20260812 14:51:30.156219 21417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:30.156541 21417 log.cc:824] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:30.157331 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64: No bootstrap required, opened a new log
I20260812 14:51:30.158520 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21290
I20260812 14:51:30.158651 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:30.158874 13898 external_mini_cluster.cc:1428] Setting key 01c9a1eb2d86161ddc34fdd01ba754d8
I20260812 14:51:30.159435 21417 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260812 14:51:30.159655 21417 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:30.159734 21417 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3b18647dc4240e6af1666571f32ca64, State: Initialized, Role: FOLLOWER
I20260812 14:51:30.159869 21417 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260812 14:51:30.159947 21417 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:30.159970 21417 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:30.160037 21417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:30.161247 21417 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260812 14:51:30.161361 21417 leader_election.cc:302] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3b18647dc4240e6af1666571f32ca64; no voters:
I20260812 14:51:30.161552 21417 leader_election.cc:288] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:30.161597 21428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:30.161782 21428 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [term 1 LEADER]: Becoming Leader. State: Replica: f3b18647dc4240e6af1666571f32ca64, State: Running, Role: LEADER
I20260812 14:51:30.161818 21417 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:30.161895 21428 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260812 14:51:30.162199 21428 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3b18647dc4240e6af1666571f32ca64. Latest consensus state: current_term: 1 leader_uuid: "f3b18647dc4240e6af1666571f32ca64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } } }
I20260812 14:51:30.162434 21428 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:30.162343 21430 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3b18647dc4240e6af1666571f32ca64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3b18647dc4240e6af1666571f32ca64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } } }
I20260812 14:51:30.162853 21430 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:30.163036 21445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:30.163570 21445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:30.165405 21445 catalog_manager.cc:1347] Generated new cluster ID: 067b30d101d64d8e9d702b4f15466b3e
I20260812 14:51:30.165511 21445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:30 dist-test-slave-8hhm krb5kdc[21267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:51:30.171140 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:30.159451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:39477 (local address 127.0.0.1:36699)
0812 14:51:30.159651 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:30.159660 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:30.159694 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:51:30.159820 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:30.159828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:30.159885 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:30.160035 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:30.160940 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:30.162294 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:30.163445 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:30.164151 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:30.164261 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:30.167306 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:30.167310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:30.167310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:30.167323 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:30.169357 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:30.169835 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:30.169838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:30.169838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:30.169969 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:30.170245 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:30.170246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:30.170246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:30.170548 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:30.170703 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:30.170844 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 14:51:30.170945 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260812 14:51:30.172293 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:30.176223 21445 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:30.176792 21445 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:30.182226 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:30.174777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:55351 (local address 127.0.0.1:36699)
0812 14:51:30.174824 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:30.174826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:30.174969 (+ 143us) server_negotiation.cc:407] Connection header received
0812 14:51:30.175019 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:30.175020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:30.175031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:30.175089 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:30.175754 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:30.176461 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:30.177584 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:30.177878 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:30.177942 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:30.179377 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:30.179378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:30.179378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:30.179384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:30.180941 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:30.181385 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:30.181386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:30.181386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:30.181445 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:30.181703 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:30.181704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:30.181704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:30.181821 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:30.181912 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:30.182072 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:51:30.182137 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:51:30.184312 21445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3b18647dc4240e6af1666571f32ca64: Generated new TSK 0
I20260812 14:51:30.184485 21445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:30.189946 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21290
W20260812 14:51:30.211973 21265 connection.cc:535] client connection to 127.0.0.1:36699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36699): Connection reset by peer (error 104)
2026-08-12T14:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1612 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:30 dist-test-slave-8hhm krb5kdc[21469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:30 dist-test-slave-8hhm krb5kdc[21469](info): set up 2 sockets
Aug 12 14:51:30 dist-test-slave-8hhm krb5kdc[21469](info): commencing operation
krb5kdc: starting...
W20260812 14:51:31.531418 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:31 dist-test-slave-8hhm krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:31.661010 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:31.748086 21490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:31.748353 21490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:31.748396 21490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:31.749552 21490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:31.749624 21490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:31.749646 21490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:31.749666 21490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:31.749696 21490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:31.751178 21490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:31.751461 21490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:31.751686 21490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:31.756651 21506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:31.756680 21513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:31.756667 21499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:31.756908 21490 server_base.cc:1034] running on GCE node
I20260812 14:51:31.757187 21490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:31.757525 21490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:31.758672 21490 hybrid_clock.cc:648] HybridClock initialized: now 1786546291758652 us; error 37 us; skew 500 ppm
Aug 12 14:51:31 dist-test-slave-8hhm krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546291, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:31.761199 21490 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:31.761720 21490 webserver.cc:460] Webserver started at http://127.0.0.1:38307/ using document root <none> and password file <none>
I20260812 14:51:31.761945 21490 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:31.762012 21490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:31.762178 21490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:31.763230 21490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "cc56e4053ba949ca8d39d8f7edba9029"
format_stamp: "Formatted at 2026-08-12 14:51:31 on dist-test-slave-8hhm"
server_key: "4b6fc2b92f826c03b47396fc06b2f8b4"
server_key_iv: "f82e6fd437a2ab2c973987ab51ff4a8e"
server_key_version: "encryptionkey@0"
I20260812 14:51:31.763608 21490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "cc56e4053ba949ca8d39d8f7edba9029"
format_stamp: "Formatted at 2026-08-12 14:51:31 on dist-test-slave-8hhm"
server_key: "4b6fc2b92f826c03b47396fc06b2f8b4"
server_key_iv: "f82e6fd437a2ab2c973987ab51ff4a8e"
server_key_version: "encryptionkey@0"
I20260812 14:51:31.764959 21490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 14:51:31.765775 21523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:31.765933 21490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:31.766044 21490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "cc56e4053ba949ca8d39d8f7edba9029"
format_stamp: "Formatted at 2026-08-12 14:51:31 on dist-test-slave-8hhm"
server_key: "4b6fc2b92f826c03b47396fc06b2f8b4"
server_key_iv: "f82e6fd437a2ab2c973987ab51ff4a8e"
server_key_version: "encryptionkey@0"
I20260812 14:51:31.766140 21490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:31.778961 21490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:31.781419 21490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:31.781585 21490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:31.785723 21490 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36751
I20260812 14:51:31.785758 21628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36751 every 8 connection(s)
I20260812 14:51:31.786119 21490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:31.786701 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21490
I20260812 14:51:31.786845 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:31.787093 13898 external_mini_cluster.cc:1428] Setting key 6145e89305a846299e59bcd62c98d29e
I20260812 14:51:31.787246 21629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:31.790261 21629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029: Bootstrap starting.
I20260812 14:51:31.790871 21629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:31.791185 21629 log.cc:824] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:31.791908 21629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029: No bootstrap required, opened a new log
I20260812 14:51:31.792696 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:31.787662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:53559 (local address 127.0.0.1:36751)
0812 14:51:31.787877 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:31.787884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:31.787910 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:51:31.788420 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:31.788427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:31.788488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:31.788646 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:31.788874 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:31.789852 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:31.790739 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:31.791195 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:31.791327 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:31.792061 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:31.792065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:31.792066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:31.792261 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:31.792291 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:31.792302 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:31.792397 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:31.792481 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:51:31.792509 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260812 14:51:31.793852 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:31.794320 21629 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260812 14:51:31.794485 21629 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:31.794533 21629 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc56e4053ba949ca8d39d8f7edba9029, State: Initialized, Role: FOLLOWER
I20260812 14:51:31.794660 21629 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260812 14:51:31.794752 21629 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:31.794801 21629 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:31.794862 21629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:31.795620 21629 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260812 14:51:31.795750 21629 leader_election.cc:302] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc56e4053ba949ca8d39d8f7edba9029; no voters:
I20260812 14:51:31.795917 21629 leader_election.cc:288] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:31.795974 21645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:31.796130 21645 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [term 1 LEADER]: Becoming Leader. State: Replica: cc56e4053ba949ca8d39d8f7edba9029, State: Running, Role: LEADER
I20260812 14:51:31.796195 21629 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:31.796217 21645 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260812 14:51:31.796545 21645 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc56e4053ba949ca8d39d8f7edba9029. Latest consensus state: current_term: 1 leader_uuid: "cc56e4053ba949ca8d39d8f7edba9029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } } }
I20260812 14:51:31.796634 21645 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:31.796550 21648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc56e4053ba949ca8d39d8f7edba9029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc56e4053ba949ca8d39d8f7edba9029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } } }
I20260812 14:51:31.796839 21648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:31.796844 21654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:31.797791 21654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:31.798904 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:31.795268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:42679 (local address 127.0.0.1:36751)
0812 14:51:31.795323 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:31.795325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:31.795451 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:51:31.795530 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:31.795531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:31.795541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:31.795600 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:31.795757 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:31.796408 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:31.797495 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:31.797806 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:31.797872 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:31.798632 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:31.798633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:31.798634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:31.798638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:31.798654 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:31.798659 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:31.798729 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:31.798817 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:51:31.798828 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:51:31.799469 21654 catalog_manager.cc:1347] Generated new cluster ID: f723710a5c4548b5b036f1e3a1748590
I20260812 14:51:31.799527 21654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:31.815115 21654 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:31.815717 21654 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:31.821511 21654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc56e4053ba949ca8d39d8f7edba9029: Generated new TSK 0
I20260812 14:51:31.821740 21654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:31.826555 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21490
W20260812 14:51:31.842979 21467 connection.cc:535] client connection to 127.0.0.1:36751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36751): Connection reset by peer (error 104)
2026-08-12T14:51:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:31 dist-test-slave-8hhm krb5kdc[21674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:31 dist-test-slave-8hhm krb5kdc[21674](info): set up 2 sockets
Aug 12 14:51:31 dist-test-slave-8hhm krb5kdc[21674](info): commencing operation
krb5kdc: starting...
W20260812 14:51:33.152865 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:33 dist-test-slave-8hhm krb5kdc[21674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:51:33.280577 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:33.364719 21702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:33.364972 21702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:33.365010 21702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:33.366168 21702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:33.366246 21702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:33.366271 21702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:33.366292 21702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:33.366322 21702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:33.367825 21702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:33.368120 21702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:33.368361 21702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:33.373435 21707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:33.373469 21710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:33.373440 21708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:33.373658 21702 server_base.cc:1034] running on GCE node
I20260812 14:51:33.374034 21702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:33.374303 21702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:33.375449 21702 hybrid_clock.cc:648] HybridClock initialized: now 1786546293375427 us; error 44 us; skew 500 ppm
Aug 12 14:51:33 dist-test-slave-8hhm krb5kdc[21674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546293, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:33.378033 21702 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:51:33.378465 21702 webserver.cc:460] Webserver started at http://127.0.0.1:46205/ using document root <none> and password file <none>
I20260812 14:51:33.378728 21702 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:33.378774 21702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:33.378947 21702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:33.380062 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f"
format_stamp: "Formatted at 2026-08-12 14:51:33 on dist-test-slave-8hhm"
server_key: "666d7e3844c8cfe802fea5d861f3d7c9"
server_key_iv: "6299d0ef486c332a6b0b8c709b8edb9b"
server_key_version: "encryptionkey@0"
I20260812 14:51:33.380439 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f"
format_stamp: "Formatted at 2026-08-12 14:51:33 on dist-test-slave-8hhm"
server_key: "666d7e3844c8cfe802fea5d861f3d7c9"
server_key_iv: "6299d0ef486c332a6b0b8c709b8edb9b"
server_key_version: "encryptionkey@0"
I20260812 14:51:33.381879 21702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:33.382669 21722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:33.382850 21702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:51:33.382964 21702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f"
format_stamp: "Formatted at 2026-08-12 14:51:33 on dist-test-slave-8hhm"
server_key: "666d7e3844c8cfe802fea5d861f3d7c9"
server_key_iv: "6299d0ef486c332a6b0b8c709b8edb9b"
server_key_version: "encryptionkey@0"
I20260812 14:51:33.383045 21702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:33.386520 21702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:33.388527 21702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:33.388698 21702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:33.392724 21702 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33713
I20260812 14:51:33.392776 21820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33713 every 8 connection(s)
I20260812 14:51:33.393134 21702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:33.394202 21821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:33.396212 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21702
I20260812 14:51:33.396355 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:33.396557 13898 external_mini_cluster.cc:1428] Setting key 4c4754126ee2e5c228d48ff24bd9fde3
I20260812 14:51:33.396865 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: Bootstrap starting.
I20260812 14:51:33.397640 21821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:33.397949 21821 log.cc:824] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:33.398720 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: No bootstrap required, opened a new log
I20260812 14:51:33.399519 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:33.397116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:56737 (local address 127.0.0.1:33713)
0812 14:51:33.397360 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:33.397376 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:33.397567 (+ 191us) server_negotiation.cc:407] Connection header received
0812 14:51:33.398169 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:33.398178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:33.398209 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:33.398357 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:33.399072 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:33.399076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:33.399076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:33.399081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:33.399129 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:33.399144 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:33.399241 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:33.399280 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:51:33.399324 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260812 14:51:33.400731 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:33.401444 21821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260812 14:51:33.401660 21821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:33.401701 21821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5021aa9fd0204eb29d9a1a6fc0b1a05f, State: Initialized, Role: FOLLOWER
I20260812 14:51:33.401824 21821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260812 14:51:33.401928 21821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:33.401969 21821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:33.402017 21821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:33.402746 21821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260812 14:51:33.402874 21821 leader_election.cc:302] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5021aa9fd0204eb29d9a1a6fc0b1a05f; no voters:
I20260812 14:51:33.403043 21821 leader_election.cc:288] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:33.403067 21845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:33.403237 21845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [term 1 LEADER]: Becoming Leader. State: Replica: 5021aa9fd0204eb29d9a1a6fc0b1a05f, State: Running, Role: LEADER
I20260812 14:51:33.403401 21821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:33.403671 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:33.402495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:35125 (local address 127.0.0.1:33713)
0812 14:51:33.402536 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:33.402538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:33.402656 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:51:33.402730 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:33.402731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:33.402735 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:33.402791 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:33.403433 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:33.403434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:33.403434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:33.403439 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:33.403453 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:33.403457 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:33.403503 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:33.403585 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:51:33.403602 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:51:33.403420 21845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } }
I20260812 14:51:33.405099 21845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5021aa9fd0204eb29d9a1a6fc0b1a05f. Latest consensus state: current_term: 1 leader_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } } }
I20260812 14:51:33.405189 21845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:33.405593 21846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5021aa9fd0204eb29d9a1a6fc0b1a05f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33713 } } }
I20260812 14:51:33.405699 21846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f [sys.catalog]: This master's current role is: LEADER
W20260812 14:51:33.405994 21871 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:33.406050 21871 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:33.406265 21872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:33.406440 21872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:33.407924 21872 catalog_manager.cc:1347] Generated new cluster ID: 94cb06e9703d4e7aa49d6eae0c1e816c
I20260812 14:51:33.407980 21872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:33.429816 21872 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:33.430013 21872 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:33.432487 21872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5021aa9fd0204eb29d9a1a6fc0b1a05f: Generated new TSK 0
I20260812 14:51:33.432682 21872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:33.440694 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21702
2026-08-12T14:51:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:33 dist-test-slave-8hhm krb5kdc[21883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:33 dist-test-slave-8hhm krb5kdc[21883](info): set up 2 sockets
Aug 12 14:51:33 dist-test-slave-8hhm krb5kdc[21883](info): commencing operation
krb5kdc: starting...
W20260812 14:51:34.768461 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:34 dist-test-slave-8hhm krb5kdc[21883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:34.900419 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41081
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:41081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:34.984968 21907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:34.985203 21907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:34.985245 21907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:34.986431 21907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:34.986478 21907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:34.986492 21907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:34.986505 21907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:34.986517 21907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:34.988008 21907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:41081
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:34.988252 21907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:34.988504 21907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:34.993620 21917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:34.993690 21907 server_base.cc:1034] running on GCE node
W20260812 14:51:34.993669 21916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:34.993620 21919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:34.994051 21907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:34.994326 21907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:34.995508 21907 hybrid_clock.cc:648] HybridClock initialized: now 1786546294995480 us; error 48 us; skew 500 ppm
Aug 12 14:51:34 dist-test-slave-8hhm krb5kdc[21883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546294, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:34.998066 21907 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:34.998564 21907 webserver.cc:460] Webserver started at http://127.13.146.190:43083/ using document root <none> and password file <none>
I20260812 14:51:34.998819 21907 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:34.998890 21907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:34.999054 21907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:35.000195 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "6202fb3ffea84ab9a96fe48046f8395c"
format_stamp: "Formatted at 2026-08-12 14:51:34 on dist-test-slave-8hhm"
server_key: "8e9c44721deb40f97ad60fbfd45d8578"
server_key_iv: "998cdea19a9c19f7395c430a515e19db"
server_key_version: "encryptionkey@0"
I20260812 14:51:35.000602 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "6202fb3ffea84ab9a96fe48046f8395c"
format_stamp: "Formatted at 2026-08-12 14:51:34 on dist-test-slave-8hhm"
server_key: "8e9c44721deb40f97ad60fbfd45d8578"
server_key_iv: "998cdea19a9c19f7395c430a515e19db"
server_key_version: "encryptionkey@0"
I20260812 14:51:35.002514 21907 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 14:51:35.003335 21933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:35.003522 21907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:35.003633 21907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "6202fb3ffea84ab9a96fe48046f8395c"
format_stamp: "Formatted at 2026-08-12 14:51:34 on dist-test-slave-8hhm"
server_key: "8e9c44721deb40f97ad60fbfd45d8578"
server_key_iv: "998cdea19a9c19f7395c430a515e19db"
server_key_version: "encryptionkey@0"
I20260812 14:51:35.003731 21907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:35.026146 21907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:35.028188 21907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:35.028376 21907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:35.032791 21907 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:41081
I20260812 14:51:35.032831 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:41081 every 8 connection(s)
I20260812 14:51:35.033298 21907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:35.034325 22030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:35.036278 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 21907
I20260812 14:51:35.036417 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:35.036615 13898 external_mini_cluster.cc:1428] Setting key a4b66e5837c16ad350fc2595fe77af52
I20260812 14:51:35.037196 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c: Bootstrap starting.
I20260812 14:51:35.037879 22030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:35.038270 22030 log.cc:824] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:35.039095 22030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c: No bootstrap required, opened a new log
I20260812 14:51:35.041419 22030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } }
I20260812 14:51:35.041587 22030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:35.041615 22030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6202fb3ffea84ab9a96fe48046f8395c, State: Initialized, Role: FOLLOWER
I20260812 14:51:35.041772 22030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } }
I20260812 14:51:35.041870 22030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:35.041909 22030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:35.041967 22030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:35.042647 22030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } }
I20260812 14:51:35.042745 22030 leader_election.cc:302] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6202fb3ffea84ab9a96fe48046f8395c; no voters:
I20260812 14:51:35.042965 22030 leader_election.cc:288] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:35.042986 22039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:35.043112 22039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [term 1 LEADER]: Becoming Leader. State: Replica: 6202fb3ffea84ab9a96fe48046f8395c, State: Running, Role: LEADER
I20260812 14:51:35.043202 22039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } }
I20260812 14:51:35.043262 22030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:35.043537 22040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6202fb3ffea84ab9a96fe48046f8395c. Latest consensus state: current_term: 1 leader_uuid: "6202fb3ffea84ab9a96fe48046f8395c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } } }
I20260812 14:51:35.043629 22040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:35.043807 22041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6202fb3ffea84ab9a96fe48046f8395c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6202fb3ffea84ab9a96fe48046f8395c" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 41081 } } }
I20260812 14:51:35.043864 22046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 14:51:35 dist-test-slave-8hhm krb5kdc[21883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:35.043880 22041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:35.044610 22046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:35.046914 22046 catalog_manager.cc:1347] Generated new cluster ID: cfe1cbc69bf04450983334ffeb24b29a
I20260812 14:51:35.046968 22046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:35.048808 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:35.037234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38089 (local address 127.13.146.190:41081)
0812 14:51:35.037423 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:35.037439 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:35.037466 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:51:35.037892 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:35.037899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:35.037945 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:35.038112 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:35.038947 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:35.039892 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:35.041075 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:35.041566 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:35.041699 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:35.044692 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:35.044696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:35.044697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:35.044710 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:35.046828 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:35.047317 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:35.047321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:35.047321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:35.047430 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:35.047712 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:35.047713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:35.047713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:35.048024 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:35.048155 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:35.048465 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 14:51:35.048576 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260812 14:51:35.050233 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:35.059484 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:35.051950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46647 (local address 127.13.146.190:41081)
0812 14:51:35.052013 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:35.052015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:35.052022 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:51:35.052041 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:35.052042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:35.052051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:35.052100 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:35.052785 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:35.053603 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:35.054672 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:35.054978 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:35.055040 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:35.056538 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:35.056540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:35.056540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:35.056547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:35.058192 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:35.058639 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:35.058640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:35.058640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:35.058708 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:35.058946 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:35.058947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:35.058947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:35.059066 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:35.059157 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:35.059328 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 14:51:35.059391 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:51:35.087338 22046 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:35.087949 22046 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:35.093683 22046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6202fb3ffea84ab9a96fe48046f8395c: Generated new TSK 0
I20260812 14:51:35.093881 22046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:35.103870 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 21907
W20260812 14:51:35.120798 21880 connection.cc:535] client connection to 127.13.146.190:41081 recv error: Network error: failed to read from TLS socket (remote: 127.13.146.190:41081): Connection reset by peer (error 104)
2026-08-12T14:51:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:35 dist-test-slave-8hhm krb5kdc[22092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:35 dist-test-slave-8hhm krb5kdc[22092](info): set up 2 sockets
Aug 12 14:51:35 dist-test-slave-8hhm krb5kdc[22092](info): commencing operation
krb5kdc: starting...
W20260812 14:51:36.460413 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:36 dist-test-slave-8hhm krb5kdc[22092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:36.587841 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36171
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:36171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:36.672742 22115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:36.672986 22115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:36.673008 22115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:36.674211 22115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:36.674256 22115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:36.674270 22115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:36.674283 22115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:36.674340 22115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:36.675854 22115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:36171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:36171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:36.676134 22115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:36.676380 22115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:36.681455 22125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:36.681725 22115 server_base.cc:1034] running on GCE node
W20260812 14:51:36.681478 22122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:36.681555 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:36.682085 22115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:36.682453 22115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:36.683625 22115 hybrid_clock.cc:648] HybridClock initialized: now 1786546296683586 us; error 60 us; skew 500 ppm
Aug 12 14:51:36 dist-test-slave-8hhm krb5kdc[22092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546296, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:36.686818 22115 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:36.687429 22115 webserver.cc:460] Webserver started at http://127.13.146.190:41975/ using document root <none> and password file <none>
I20260812 14:51:36.687741 22115 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:36.687810 22115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:36.687964 22115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:36.689572 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d"
format_stamp: "Formatted at 2026-08-12 14:51:36 on dist-test-slave-8hhm"
server_key: "a2bfee6b62419e49a883d7449a97804b"
server_key_iv: "1127944668c6d8eaca33f443715815a2"
server_key_version: "encryptionkey@0"
I20260812 14:51:36.689950 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d"
format_stamp: "Formatted at 2026-08-12 14:51:36 on dist-test-slave-8hhm"
server_key: "a2bfee6b62419e49a883d7449a97804b"
server_key_iv: "1127944668c6d8eaca33f443715815a2"
server_key_version: "encryptionkey@0"
I20260812 14:51:36.692180 22115 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 14:51:36.694169 22136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:36.694381 22115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260812 14:51:36.694490 22115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d"
format_stamp: "Formatted at 2026-08-12 14:51:36 on dist-test-slave-8hhm"
server_key: "a2bfee6b62419e49a883d7449a97804b"
server_key_iv: "1127944668c6d8eaca33f443715815a2"
server_key_version: "encryptionkey@0"
I20260812 14:51:36.694584 22115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:36.717953 22115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:36.719918 22115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:36.720088 22115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:36.724231 22115 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:36171
I20260812 14:51:36.724287 22228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:36171 every 8 connection(s)
I20260812 14:51:36.724645 22115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:36.725656 22229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:36.728076 22229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: Bootstrap starting.
I20260812 14:51:36.728642 22229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:36.728901 22229 log.cc:824] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:36.729727 22229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: No bootstrap required, opened a new log
I20260812 14:51:36.731361 22229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } }
I20260812 14:51:36.731513 22229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:36.731536 22229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b16f0fc9b99494f8db3f04f0ad75f0d, State: Initialized, Role: FOLLOWER
I20260812 14:51:36.731678 22229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } }
I20260812 14:51:36.731776 22229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:36.731801 22229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:36.731832 22229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:36.732739 22229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } }
I20260812 14:51:36.732836 22229 leader_election.cc:302] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b16f0fc9b99494f8db3f04f0ad75f0d; no voters:
I20260812 14:51:36.732964 22229 leader_election.cc:288] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:36.733049 22232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:36.733251 22229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:36.733255 22232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [term 1 LEADER]: Becoming Leader. State: Replica: 5b16f0fc9b99494f8db3f04f0ad75f0d, State: Running, Role: LEADER
I20260812 14:51:36.733479 22232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } }
I20260812 14:51:36.733996 22234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } } }
I20260812 14:51:36.733971 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b16f0fc9b99494f8db3f04f0ad75f0d. Latest consensus state: current_term: 1 leader_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b16f0fc9b99494f8db3f04f0ad75f0d" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 36171 } } }
I20260812 14:51:36.734191 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:36.734095 22234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:36.734692 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22115
I20260812 14:51:36.734848 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:36.735057 13898 external_mini_cluster.cc:1428] Setting key 8895c441486bb46382a9fd6eb0bdaa61
W20260812 14:51:36.736040 22265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:36.736102 22265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:36.736191 22252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:36.736390 22252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:36.738574 22252 catalog_manager.cc:1347] Generated new cluster ID: 07b80518293e4f57970205b12cc9daa4
I20260812 14:51:36.738653 22252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:36.741614 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:36.735882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54197 (local address 127.13.146.190:36171)
0812 14:51:36.736246 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:36.736257 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:36.736293 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:51:36.736436 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:36.736444 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:36.736507 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:36.736663 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:36.736950 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:36.738153 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:36.739301 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:36.739911 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:36.740021 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:36.740865 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:36.740869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:36.740869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:36.741074 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:36.741113 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:36.741127 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:36.741250 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:36.741377 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:51:36.741403 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":78,"threads_started":1}
I20260812 14:51:36.742906 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:36.750526 22252 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:36.751046 22252 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:36.753042 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:36.745613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42253 (local address 127.13.146.190:36171)
0812 14:51:36.745658 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:36.745660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:36.749429 (+ 3769us) server_negotiation.cc:407] Connection header received
0812 14:51:36.749470 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:36.749471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:36.749493 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:36.749570 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:36.749724 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:36.750473 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:36.751571 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:36.751890 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:36.751951 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:36.752768 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:36.752770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:36.752770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:36.752774 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:36.752790 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:36.752795 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:36.752843 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:36.752955 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:51:36.752968 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:51:36.761034 22252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b16f0fc9b99494f8db3f04f0ad75f0d: Generated new TSK 0
I20260812 14:51:36.761284 22252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:36.766260 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22115
W20260812 14:51:36.788115 22086 connection.cc:535] client connection to 127.13.146.190:36171 recv error: Network error: failed to read from TLS socket (remote: 127.13.146.190:36171): Connection reset by peer (error 104)
2026-08-12T14:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:36 dist-test-slave-8hhm krb5kdc[22287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:36 dist-test-slave-8hhm krb5kdc[22287](info): set up 2 sockets
Aug 12 14:51:36 dist-test-slave-8hhm krb5kdc[22287](info): commencing operation
krb5kdc: starting...
W20260812 14:51:38.087435 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:38 dist-test-slave-8hhm krb5kdc[22287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:38.216776 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35995
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:35995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:38.303658 22311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:38.303925 22311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:38.303967 22311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:38.305266 22311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:38.305342 22311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:38.305368 22311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:38.305400 22311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:38.305428 22311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:38.307034 22311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:35995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:35995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:38.307320 22311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:38.307560 22311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:38.312847 22318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:38.312865 22321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:38.312974 22319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:38.313081 22311 server_base.cc:1034] running on GCE node
I20260812 14:51:38.313467 22311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:38.313751 22311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:38.314924 22311 hybrid_clock.cc:648] HybridClock initialized: now 1786546298314902 us; error 67 us; skew 500 ppm
Aug 12 14:51:38 dist-test-slave-8hhm krb5kdc[22287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546298, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:38.317682 22311 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:38.318260 22311 webserver.cc:460] Webserver started at http://127.13.146.190:46627/ using document root <none> and password file <none>
I20260812 14:51:38.318514 22311 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:38.318583 22311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:38.318727 22311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:38.319952 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "33fef08233bc4062810c6ce6ed92e911"
format_stamp: "Formatted at 2026-08-12 14:51:38 on dist-test-slave-8hhm"
server_key: "fdc06219bd7e962954a7501c88c3fe08"
server_key_iv: "87207194131fdd63bd711f2b34ae1649"
server_key_version: "encryptionkey@0"
I20260812 14:51:38.320356 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "33fef08233bc4062810c6ce6ed92e911"
format_stamp: "Formatted at 2026-08-12 14:51:38 on dist-test-slave-8hhm"
server_key: "fdc06219bd7e962954a7501c88c3fe08"
server_key_iv: "87207194131fdd63bd711f2b34ae1649"
server_key_version: "encryptionkey@0"
I20260812 14:51:38.321869 22311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:38.322742 22333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:38.322925 22311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:38.323055 22311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "33fef08233bc4062810c6ce6ed92e911"
format_stamp: "Formatted at 2026-08-12 14:51:38 on dist-test-slave-8hhm"
server_key: "fdc06219bd7e962954a7501c88c3fe08"
server_key_iv: "87207194131fdd63bd711f2b34ae1649"
server_key_version: "encryptionkey@0"
I20260812 14:51:38.323134 22311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:38.326665 22311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:38.329063 22311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:38.329293 22311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:38.333599 22311 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:35995
I20260812 14:51:38.333633 22423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:35995 every 8 connection(s)
I20260812 14:51:38.333977 22311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:38.334997 22425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:38.337445 22425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911: Bootstrap starting.
I20260812 14:51:38.338076 22425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:38.338377 22425 log.cc:824] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:38.339110 22425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911: No bootstrap required, opened a new log
I20260812 14:51:38.340862 22425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } }
I20260812 14:51:38.341019 22425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:38.341043 22425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fef08233bc4062810c6ce6ed92e911, State: Initialized, Role: FOLLOWER
I20260812 14:51:38.341252 22425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } }
I20260812 14:51:38.341336 22425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:38.341359 22425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:38.341430 22425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:38.342347 22425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } }
I20260812 14:51:38.342442 22425 leader_election.cc:302] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33fef08233bc4062810c6ce6ed92e911; no voters:
I20260812 14:51:38.342675 22425 leader_election.cc:288] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:38.342773 22432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:38.342973 22432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [term 1 LEADER]: Becoming Leader. State: Replica: 33fef08233bc4062810c6ce6ed92e911, State: Running, Role: LEADER
I20260812 14:51:38.342998 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22311
I20260812 14:51:38.343070 22432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } }
I20260812 14:51:38.343127 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:38.343323 13898 external_mini_cluster.cc:1428] Setting key d7ea48339754bc037e8d7a36a2e9d422
I20260812 14:51:38.343467 22436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33fef08233bc4062810c6ce6ed92e911. Latest consensus state: current_term: 1 leader_uuid: "33fef08233bc4062810c6ce6ed92e911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } } }
I20260812 14:51:38.343561 22436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:38.343676 22425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:38.343694 22436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33fef08233bc4062810c6ce6ed92e911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33fef08233bc4062810c6ce6ed92e911" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 35995 } } }
I20260812 14:51:38.343750 22436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:38.343876 22443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:38.344452 22443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:38.346335 22443 catalog_manager.cc:1347] Generated new cluster ID: aa54735883c54acab6438ce03cc74dca
I20260812 14:51:38.346392 22443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:38.350265 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:38.344024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54661 (local address 127.13.146.190:35995)
0812 14:51:38.344431 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:38.344442 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:38.344650 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:51:38.344827 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:38.344836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:38.344870 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:38.345397 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:38.349788 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:38.349792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:38.349792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:38.349797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:38.349854 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:38.349866 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:38.349939 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:38.350016 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:51:38.350062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":79,"threads_started":1}
I20260812 14:51:38.351290 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:38.362792 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:38.353591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36331 (local address 127.13.146.190:35995)
0812 14:51:38.353630 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:38.353632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:38.361486 (+ 7854us) server_negotiation.cc:407] Connection header received
0812 14:51:38.361615 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:38.361617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:38.361624 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:38.361687 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:38.362498 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:38.362499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:38.362500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:38.362506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:38.362525 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:38.362530 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:38.362595 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:38.362654 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:51:38.362672 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:51:38.373765 22443 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:38.373994 22443 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:38.382743 22443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33fef08233bc4062810c6ce6ed92e911: Generated new TSK 0
I20260812 14:51:38.382987 22443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:38.391808 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22311
2026-08-12T14:51:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:38 dist-test-slave-8hhm krb5kdc[22481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:38 dist-test-slave-8hhm krb5kdc[22481](info): set up 2 sockets
Aug 12 14:51:38 dist-test-slave-8hhm krb5kdc[22481](info): commencing operation
krb5kdc: starting...
W20260812 14:51:39.720748 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:39 dist-test-slave-8hhm krb5kdc[22481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:39.847810 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:33999
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:33999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:39.932211 22502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:39.932427 22502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:39.932451 22502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:39.933586 22502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:39.933622 22502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:39.933635 22502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:39.933647 22502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:39.933658 22502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:39.935066 22502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:33999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:33999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:39.935272 22502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:39.935436 22502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:39.940478 22512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:39.940534 22502 server_base.cc:1034] running on GCE node
W20260812 14:51:39.940516 22518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:39.940493 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:39.940909 22502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:39.941318 22502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:39.942463 22502 hybrid_clock.cc:648] HybridClock initialized: now 1786546299942438 us; error 45 us; skew 500 ppm
Aug 12 14:51:39 dist-test-slave-8hhm krb5kdc[22481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546299, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:39.951429 22502 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:39.951985 22502 webserver.cc:460] Webserver started at http://127.13.146.190:45111/ using document root <none> and password file <none>
I20260812 14:51:39.952235 22502 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:39.952284 22502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:39.952426 22502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:39.953670 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea"
format_stamp: "Formatted at 2026-08-12 14:51:39 on dist-test-slave-8hhm"
server_key: "1bd59b92529e65d1b04a2950b0fcaaf1"
server_key_iv: "9d91d89f68e4df2894d284c32b0f2a9d"
server_key_version: "encryptionkey@0"
I20260812 14:51:39.954054 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea"
format_stamp: "Formatted at 2026-08-12 14:51:39 on dist-test-slave-8hhm"
server_key: "1bd59b92529e65d1b04a2950b0fcaaf1"
server_key_iv: "9d91d89f68e4df2894d284c32b0f2a9d"
server_key_version: "encryptionkey@0"
I20260812 14:51:39.955478 22502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:51:39.956298 22531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:39.956507 22502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:39.956591 22502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea"
format_stamp: "Formatted at 2026-08-12 14:51:39 on dist-test-slave-8hhm"
server_key: "1bd59b92529e65d1b04a2950b0fcaaf1"
server_key_iv: "9d91d89f68e4df2894d284c32b0f2a9d"
server_key_version: "encryptionkey@0"
I20260812 14:51:39.956687 22502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:39.963270 22502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:39.965662 22502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:39.965821 22502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:39.970300 22502 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:33999
I20260812 14:51:39.970396 22621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:33999 every 8 connection(s)
I20260812 14:51:39.970736 22502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:39.971721 22627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:39.973774 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22502
I20260812 14:51:39.973920 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:39.974174 13898 external_mini_cluster.cc:1428] Setting key 31ffb1b878b44ffb9a60037a9ad680db
I20260812 14:51:39.974754 22627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea: Bootstrap starting.
I20260812 14:51:39.975447 22627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:39.975811 22627 log.cc:824] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:39.976603 22627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea: No bootstrap required, opened a new log
I20260812 14:51:39.978663 22627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } }
I20260812 14:51:39.978838 22627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:39.978893 22627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160c9e1f42c34a64ac5fe00b6ca1f6ea, State: Initialized, Role: FOLLOWER
I20260812 14:51:39.979048 22627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } }
I20260812 14:51:39.979152 22627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:39.979240 22627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:39.979305 22627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:39.979897 22627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } }
I20260812 14:51:39.980026 22627 leader_election.cc:302] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 160c9e1f42c34a64ac5fe00b6ca1f6ea; no voters:
I20260812 14:51:39.980240 22627 leader_election.cc:288] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:39.980261 22639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:39.980439 22639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [term 1 LEADER]: Becoming Leader. State: Replica: 160c9e1f42c34a64ac5fe00b6ca1f6ea, State: Running, Role: LEADER
I20260812 14:51:39.980614 22627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:39.980589 22639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } }
Aug 12 14:51:39 dist-test-slave-8hhm krb5kdc[22481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:39.981015 22639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 160c9e1f42c34a64ac5fe00b6ca1f6ea. Latest consensus state: current_term: 1 leader_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } } }
I20260812 14:51:39.981182 22639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:39.981714 22657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:39.982416 22657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:39.982054 22643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "160c9e1f42c34a64ac5fe00b6ca1f6ea" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 33999 } } }
I20260812 14:51:39.982841 22643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:39.984733 22657 catalog_manager.cc:1347] Generated new cluster ID: aff654ad57b9400db2169e75624fb7a5
I20260812 14:51:39.984819 22657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:39.986081 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:39.974747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58549 (local address 127.13.146.190:33999)
0812 14:51:39.974962 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:39.974970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:39.974996 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:51:39.975426 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:39.975432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:39.975476 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:39.975694 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:39.976418 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:39.977409 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:39.978180 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:39.978580 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:39.978706 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:39.981498 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:39.981502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:39.981503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:39.981515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:39.983899 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:39.984354 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:39.984357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:39.984358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:39.984464 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:39.984724 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:39.984725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:39.984726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:39.985083 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:39.985240 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:39.985748 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 14:51:39.985880 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260812 14:51:39.987151 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:40.000880 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:39.988673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49395 (local address 127.13.146.190:33999)
0812 14:51:39.988714 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:39.988716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:39.993426 (+ 4710us) server_negotiation.cc:407] Connection header received
0812 14:51:39.993489 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:39.993491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:39.993502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:39.993554 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:39.994254 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:39.994975 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:39.996014 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:39.996329 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:39.996392 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:39.997895 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:39.997897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:39.997897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:39.997905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:39.999566 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:39.999987 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:39.999988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:39.999988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:40.000050 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:40.000289 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:40.000290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:40.000291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:40.000421 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:40.000547 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:40.000687 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:51:40.000771 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:51:40.000991 22657 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:40.001878 22657 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:40.006451 22657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 160c9e1f42c34a64ac5fe00b6ca1f6ea: Generated new TSK 0
I20260812 14:51:40.006659 22657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:40.016044 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22502
2026-08-12T14:51:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:40 dist-test-slave-8hhm krb5kdc[22690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:40 dist-test-slave-8hhm krb5kdc[22690](info): set up 2 sockets
Aug 12 14:51:40 dist-test-slave-8hhm krb5kdc[22690](info): commencing operation
krb5kdc: starting...
W20260812 14:51:41.401042 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:41 dist-test-slave-8hhm krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:41.532793 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34261
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:41941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:34261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:41.619074 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:41.619357 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:41.619395 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:41.620550 22718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:41.620623 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:41.620651 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:41.620682 22718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:41.620707 22718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:41.622308 22718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:41941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:34261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:41.622620 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:41.622872 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:41.628149 22735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:41.628228 22728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:41.628152 22727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:41.628522 22718 server_base.cc:1034] running on GCE node
I20260812 14:51:41.628789 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:41.629038 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:41.630229 22718 hybrid_clock.cc:648] HybridClock initialized: now 1786546301630191 us; error 67 us; skew 500 ppm
Aug 12 14:51:41 dist-test-slave-8hhm krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546301, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:41.633030 22718 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:41.633641 22718 webserver.cc:460] Webserver started at http://127.13.146.190:41683/ using document root <none> and password file <none>
I20260812 14:51:41.633906 22718 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:41.633977 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:41.634140 22718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:41.635277 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "63931f4f60c148e8a00e818a5fb5e739"
format_stamp: "Formatted at 2026-08-12 14:51:41 on dist-test-slave-8hhm"
server_key: "904cbfa1417a3f3e10cc7eeaaab2f6ca"
server_key_iv: "be5bc9010538bd886ef00b89e298054e"
server_key_version: "encryptionkey@0"
I20260812 14:51:41.635664 22718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "63931f4f60c148e8a00e818a5fb5e739"
format_stamp: "Formatted at 2026-08-12 14:51:41 on dist-test-slave-8hhm"
server_key: "904cbfa1417a3f3e10cc7eeaaab2f6ca"
server_key_iv: "be5bc9010538bd886ef00b89e298054e"
server_key_version: "encryptionkey@0"
I20260812 14:51:41.637090 22718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:41.638015 22742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:41.638265 22718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:41.638365 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "63931f4f60c148e8a00e818a5fb5e739"
format_stamp: "Formatted at 2026-08-12 14:51:41 on dist-test-slave-8hhm"
server_key: "904cbfa1417a3f3e10cc7eeaaab2f6ca"
server_key_iv: "be5bc9010538bd886ef00b89e298054e"
server_key_version: "encryptionkey@0"
I20260812 14:51:41.638465 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:41.650697 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:41.652951 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:41.653136 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:41.657737 22718 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:34261
I20260812 14:51:41.657780 22842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:34261 every 8 connection(s)
I20260812 14:51:41.658165 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:41.658952 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22718
I20260812 14:51:41.659080 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:41.659304 13898 external_mini_cluster.cc:1428] Setting key ba66958b6b5015143ae654c08098dce0
I20260812 14:51:41.659296 22843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:41.662304 22843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739: Bootstrap starting.
I20260812 14:51:41.662925 22843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:41.663234 22843 log.cc:824] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:41.663985 22843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739: No bootstrap required, opened a new log
I20260812 14:51:41.664965 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:41.659877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58241 (local address 127.13.146.190:34261)
0812 14:51:41.660091 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:41.660101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:41.660177 (+ 76us) server_negotiation.cc:407] Connection header received
0812 14:51:41.660713 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:41.660720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:41.660770 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:41.660922 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:41.661174 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:41.662111 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:41.662983 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:41.663472 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:41.663588 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:41.664339 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:41.664344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:41.664344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:41.664515 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:41.664545 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:41.664556 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:41.664648 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:41.664714 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 14:51:41.664780 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260812 14:51:41.665656 22843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } }
I20260812 14:51:41.665845 22843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:41.665927 22843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63931f4f60c148e8a00e818a5fb5e739, State: Initialized, Role: FOLLOWER
I20260812 14:51:41.666054 22843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } }
I20260812 14:51:41.666184 22843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:41.666231 22843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:41.666293 22843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:41.666307 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:41.666827 22843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } }
I20260812 14:51:41.666914 22843 leader_election.cc:302] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63931f4f60c148e8a00e818a5fb5e739; no voters:
I20260812 14:51:41.667068 22843 leader_election.cc:288] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:41.667105 22851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:41.667238 22851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [term 1 LEADER]: Becoming Leader. State: Replica: 63931f4f60c148e8a00e818a5fb5e739, State: Running, Role: LEADER
I20260812 14:51:41.667339 22843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:41.667342 22851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } }
I20260812 14:51:41.668166 22857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63931f4f60c148e8a00e818a5fb5e739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } } }
I20260812 14:51:41.668181 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63931f4f60c148e8a00e818a5fb5e739. Latest consensus state: current_term: 1 leader_uuid: "63931f4f60c148e8a00e818a5fb5e739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63931f4f60c148e8a00e818a5fb5e739" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 34261 } } }
I20260812 14:51:41.668258 22857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:41.668282 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:41.668431 22875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:41.668987 22875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:41.670652 22875 catalog_manager.cc:1347] Generated new cluster ID: ead2cc04b7d84e2e8729a69713c65efd
I20260812 14:51:41.670701 22875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:41.671289 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:41.667683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47017 (local address 127.13.146.190:34261)
0812 14:51:41.667723 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:41.667725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:41.667860 (+ 135us) server_negotiation.cc:407] Connection header received
0812 14:51:41.667951 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:41.667952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:41.667964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:41.667995 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:41.668233 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:41.668920 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:41.669920 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:41.670228 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:41.670278 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:41.671039 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:41.671040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:41.671040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:41.671045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:41.671061 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:41.671066 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:41.671133 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:41.671191 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 14:51:41.671218 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:51:41.688659 22875 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:41.689200 22875 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:41.696918 22875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63931f4f60c148e8a00e818a5fb5e739: Generated new TSK 0
I20260812 14:51:41.697182 22875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:41.706257 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22718
2026-08-12T14:51:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1692 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:41 dist-test-slave-8hhm krb5kdc[22902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:41 dist-test-slave-8hhm krb5kdc[22902](info): set up 2 sockets
Aug 12 14:51:41 dist-test-slave-8hhm krb5kdc[22902](info): commencing operation
krb5kdc: starting...
W20260812 14:51:43.031189 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:43 dist-test-slave-8hhm krb5kdc[22902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:43.159969 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45937
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:43.245817 22921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:43.246018 22921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:43.246040 22921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:43.247193 22921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:43.247236 22921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:43.247251 22921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:43.247263 22921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:43.247275 22921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:43.248795 22921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:43.249061 22921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:43.249369 22921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:51:43.254447 22921 server_base.cc:1034] running on GCE node
W20260812 14:51:43.254392 22938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:43.254386 22939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:43.254412 22941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:43.254912 22921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:43.255242 22921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:43.256393 22921 hybrid_clock.cc:648] HybridClock initialized: now 1786546303256382 us; error 32 us; skew 500 ppm
Aug 12 14:51:43 dist-test-slave-8hhm krb5kdc[22902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546303, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:43.259172 22921 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:43.259660 22921 webserver.cc:460] Webserver started at http://127.13.146.190:40901/ using document root <none> and password file <none>
I20260812 14:51:43.259909 22921 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:43.259976 22921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:43.260140 22921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:43.261307 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "f459c90835b8437a965c422382073ee4"
format_stamp: "Formatted at 2026-08-12 14:51:43 on dist-test-slave-8hhm"
server_key: "fae8e3ad179b204aa0d38bcff657ae20"
server_key_iv: "01ac5c2c00d44b9bf08538b7bf2c7f84"
server_key_version: "encryptionkey@0"
I20260812 14:51:43.261698 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "f459c90835b8437a965c422382073ee4"
format_stamp: "Formatted at 2026-08-12 14:51:43 on dist-test-slave-8hhm"
server_key: "fae8e3ad179b204aa0d38bcff657ae20"
server_key_iv: "01ac5c2c00d44b9bf08538b7bf2c7f84"
server_key_version: "encryptionkey@0"
I20260812 14:51:43.263067 22921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:51:43.263878 22956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:43.264117 22921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:51:43.264225 22921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "f459c90835b8437a965c422382073ee4"
format_stamp: "Formatted at 2026-08-12 14:51:43 on dist-test-slave-8hhm"
server_key: "fae8e3ad179b204aa0d38bcff657ae20"
server_key_iv: "01ac5c2c00d44b9bf08538b7bf2c7f84"
server_key_version: "encryptionkey@0"
I20260812 14:51:43.264302 22921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:43.267728 22921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:43.270226 22921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:43.270387 22921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:43.274631 22921 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:45937
I20260812 14:51:43.274672 23055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:45937 every 8 connection(s)
I20260812 14:51:43.275018 22921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:43.275574 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 22921
I20260812 14:51:43.275709 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:43.275995 13898 external_mini_cluster.cc:1428] Setting key d0c2c9873db10a608af9a1e5dc7d840a
I20260812 14:51:43.276415 23057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:43.278591 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:43.276569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54643 (local address 127.13.146.190:45937)
0812 14:51:43.276749 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:43.276758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:43.276800 (+ 42us) server_negotiation.cc:407] Connection header received
0812 14:51:43.277304 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:43.277312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:43.277338 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:43.277460 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:43.278118 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:43.278121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:43.278122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:43.278125 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:43.278164 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:43.278175 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:43.278230 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:43.278330 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:51:43.278391 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260812 14:51:43.279608 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:43.280265 23057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4: Bootstrap starting.
I20260812 14:51:43.281065 23057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:43.281477 23057 log.cc:824] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:43.282378 23057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4: No bootstrap required, opened a new log
I20260812 14:51:43.282490 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:43.281314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42811 (local address 127.13.146.190:45937)
0812 14:51:43.281357 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:43.281359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:43.281503 (+ 144us) server_negotiation.cc:407] Connection header received
0812 14:51:43.281586 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:43.281586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:43.281592 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:43.281647 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:43.282256 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:43.282257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:43.282257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:51:43.282261 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:43.282276 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:51:43.282279 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:43.282325 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:43.282381 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:51:43.282391 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:51:43.284324 23057 raft_consensus.cc:348] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } }
I20260812 14:51:43.284489 23057 raft_consensus.cc:374] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:51:43.284538 23057 raft_consensus.cc:729] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f459c90835b8437a965c422382073ee4, State: Initialized, Role: FOLLOWER
I20260812 14:51:43.284678 23057 consensus_queue.cc:260] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } }
I20260812 14:51:43.284765 23057 raft_consensus.cc:388] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:51:43.284811 23057 raft_consensus.cc:482] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:51:43.284860 23057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:43.285475 23057 raft_consensus.cc:504] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } }
I20260812 14:51:43.285600 23057 leader_election.cc:302] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f459c90835b8437a965c422382073ee4; no voters:
I20260812 14:51:43.285811 23057 leader_election.cc:288] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:51:43.285864 23077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:43.286048 23077 raft_consensus.cc:686] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [term 1 LEADER]: Becoming Leader. State: Replica: f459c90835b8437a965c422382073ee4, State: Running, Role: LEADER
I20260812 14:51:43.286059 23057 sys_catalog.cc:564] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:43.286158 23077 consensus_queue.cc:237] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } }
I20260812 14:51:43.286567 23081 sys_catalog.cc:455] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f459c90835b8437a965c422382073ee4. Latest consensus state: current_term: 1 leader_uuid: "f459c90835b8437a965c422382073ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } } }
I20260812 14:51:43.286718 23081 sys_catalog.cc:458] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:43.286566 23079 sys_catalog.cc:455] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f459c90835b8437a965c422382073ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f459c90835b8437a965c422382073ee4" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45937 } } }
I20260812 14:51:43.287160 23079 sys_catalog.cc:458] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:43.287297 23095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:43.287734 23095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:43.289355 23095 catalog_manager.cc:1347] Generated new cluster ID: 94c12a70edf54deabadc05568b989b8a
I20260812 14:51:43.289426 23095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:43.298918 23095 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:43.299125 23095 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:43.315478 23095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f459c90835b8437a965c422382073ee4: Generated new TSK 0
I20260812 14:51:43.315709 23095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:43.327452 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 22921
2026-08-12T14:51:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1613 ms)
[----------] 18 tests from AuthTokenIssuingTest (29676 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:43 dist-test-slave-8hhm krb5kdc[23116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:43 dist-test-slave-8hhm krb5kdc[23116](info): set up 2 sockets
Aug 12 14:51:43 dist-test-slave-8hhm krb5kdc[23116](info): commencing operation
krb5kdc: starting...
W20260812 14:51:44.634214 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:44 dist-test-slave-8hhm krb5kdc[23116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:44.764794 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45667
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:44.850186 23146 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:44.850471 23146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:44.850517 23146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:44.850577 23146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:44.851816 23146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:44.851900 23146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:44.851923 23146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:44.851946 23146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:44.851974 23146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:44.854153 23146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:45667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:44.854450 23146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:44.854694 23146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:44.859875 23154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:44.859934 23162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:44.859858 23156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:44.860143 23146 server_base.cc:1034] running on GCE node
I20260812 14:51:44.860445 23146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:44.860765 23146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:44.861936 23146 hybrid_clock.cc:648] HybridClock initialized: now 1786546304861915 us; error 42 us; skew 500 ppm
Aug 12 14:51:44 dist-test-slave-8hhm krb5kdc[23116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:44.864540 23146 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:44.865042 23146 webserver.cc:460] Webserver started at http://127.13.146.190:36187/ using document root <none> and password file <none>
I20260812 14:51:44.865358 23146 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:44.865432 23146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:44.865602 23146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:44.866717 23146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "22696f699dad4d73b4fff60f80e68e95"
format_stamp: "Formatted at 2026-08-12 14:51:44 on dist-test-slave-8hhm"
server_key: "da9f1fea16a33241bc520718a3e05eb2"
server_key_iv: "bbfe3bdf3c81faae08ad173252f36572"
server_key_version: "encryptionkey@0"
I20260812 14:51:44.867116 23146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "22696f699dad4d73b4fff60f80e68e95"
format_stamp: "Formatted at 2026-08-12 14:51:44 on dist-test-slave-8hhm"
server_key: "da9f1fea16a33241bc520718a3e05eb2"
server_key_iv: "bbfe3bdf3c81faae08ad173252f36572"
server_key_version: "encryptionkey@0"
I20260812 14:51:44.868467 23146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:44.869287 23176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:44.869468 23146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:44.869572 23146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "22696f699dad4d73b4fff60f80e68e95"
format_stamp: "Formatted at 2026-08-12 14:51:44 on dist-test-slave-8hhm"
server_key: "da9f1fea16a33241bc520718a3e05eb2"
server_key_iv: "bbfe3bdf3c81faae08ad173252f36572"
server_key_version: "encryptionkey@0"
I20260812 14:51:44.869668 23146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:44.896232 23146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:44.898294 23146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:44.898469 23146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:44.902920 23146 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:45667
I20260812 14:51:44.902961 23275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:45667 every 8 connection(s)
I20260812 14:51:44.903349 23146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:44.904354 23276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:44.906050 23276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:44.910887 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23146
I20260812 14:51:44.911033 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:44.911226 13898 external_mini_cluster.cc:1428] Setting key f0b535c03c89186b96782d3289ca7498
Aug 12 14:51:44 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:44.916281 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.906694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45667 (local address 127.0.0.1:47354)
0812 14:51:44.906925 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:44.906970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:44.907492 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:44.907796 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:44.907799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:44.908085 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:44.908247 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:44.908252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:44.909330 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:44.909331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:44.910005 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:44.910008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:44.910512 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:44.910513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:44.910592 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:44.911248 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:44.911256 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:44.912762 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:44.914699 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:44.914707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:44.914709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:44.914917 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:44.915170 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:44.915171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:44.915171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:44.915266 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:44.915640 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:44.915646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:44.915711 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 14:51:44.915816 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 14:51:44.916074 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":108,"thread_start_us":98,"threads_started":1}
I20260812 14:51:44.916292 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.906759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.13.146.190:45667)
0812 14:51:44.906937 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:44.906946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:44.907077 (+ 131us) server_negotiation.cc:407] Connection header received
0812 14:51:44.907655 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:44.907657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:44.907685 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:44.907791 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:44.908352 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:44.909181 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:44.910105 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:44.910422 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:44.910543 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:44.912856 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:44.912864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:44.912864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:44.912872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:44.914595 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:44.915003 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:44.915006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:44.915007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:44.915094 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:44.915340 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:44.915341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:44.915341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:44.915560 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:44.915650 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:44.915864 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 14:51:44.916002 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260812 14:51:44.917367 23276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:44.917759 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.917632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57880)
0812 14:51:44.917682 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:44.917729 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:51:44.917888 23178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
I20260812 14:51:44.918340 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.918241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57886)
0812 14:51:44.918290 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:44.918316 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:51:44.918452 23276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:35851: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.146.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
I20260812 14:51:44.938060 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:35851
--webserver_interface=127.13.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:51:44.967696 23276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } attempt: 1
I20260812 14:51:44.968313 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.968170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57898)
0812 14:51:44.968238 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:44.968281 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 14:51:44.968731 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:44.968588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57904)
0812 14:51:44.968651 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:44.968696 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 14:51:44.968870 23276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:35851: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
W20260812 14:51:45.022235 23290 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:45.022456 23290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:45.022475 23290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:45.022493 23290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:45.023710 23290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:45.023754 23290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:45.023768 23290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:45.023782 23290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:45.023845 23290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:45.025391 23290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:35851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:45.025651 23290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:45.025919 23290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:45.031028 23306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:45.031038 23297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:45.031028 23301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:45.031311 23290 server_base.cc:1034] running on GCE node
I20260812 14:51:45.031616 23290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:45.031949 23290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:45.033141 23290 hybrid_clock.cc:648] HybridClock initialized: now 1786546305033136 us; error 37 us; skew 500 ppm
I20260812 14:51:45.037021 23276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } attempt: 2
I20260812 14:51:45.037456 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.037336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57906)
0812 14:51:45.037391 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.037427 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:51:45.037807 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.037704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57922)
0812 14:51:45.037769 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.037792 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 14:51:45.037899 23276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:35851: Network error: Client connection negotiation failed: client connection to 127.13.146.189:35851: connect: Connection refused (error 111)
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:45.039844 23290 init.cc:377] Logged in from keytab as kudu/127.13.146.189@KRBTEST.COM (short username kudu)
I20260812 14:51:45.040410 23290 webserver.cc:460] Webserver started at http://127.13.146.189:37373/ using document root <none> and password file <none>
I20260812 14:51:45.040692 23290 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:45.040784 23290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:45.040952 23290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:45.042203 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data/instance:
uuid: "ccdf6378f1914a338bc215e9dfa87e2c"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "b3e06d8dfce54ca16229f61084d623b1"
server_key_iv: "790b69c88bd0961c0c5e2d27d6a46be7"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.042629 23290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal/instance:
uuid: "ccdf6378f1914a338bc215e9dfa87e2c"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "b3e06d8dfce54ca16229f61084d623b1"
server_key_iv: "790b69c88bd0961c0c5e2d27d6a46be7"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.044062 23290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:45.044847 23324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:45.045042 23290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:45.045152 23290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "ccdf6378f1914a338bc215e9dfa87e2c"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "b3e06d8dfce54ca16229f61084d623b1"
server_key_iv: "790b69c88bd0961c0c5e2d27d6a46be7"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.045254 23290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:45.070519 23290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:45.072808 23290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:45.072985 23290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:45.077538 23290 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.189:35851
I20260812 14:51:45.077579 23430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.189:35851 every 8 connection(s)
I20260812 14:51:45.077970 23290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
I20260812 14:51:45.078896 23431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:45.080566 23431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:45.084280 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23290
I20260812 14:51:45.084376 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-1/wal/instance
I20260812 14:51:45.084547 13898 external_mini_cluster.cc:1428] Setting key 99ca47a7d6cf668b4803dc3aaefc099b
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:45.091132 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.081325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.13.146.190:45667)
0812 14:51:45.081398 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.081400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.081650 (+ 250us) server_negotiation.cc:407] Connection header received
0812 14:51:45.082176 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.082177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.082189 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.082262 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.082967 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.083481 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.084823 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.085054 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.085119 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.087865 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.087866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.087867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.087876 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.089655 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.090054 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.090055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.090055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.090123 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.090384 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.090385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.090385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.090545 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.090628 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.090891 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.090993 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:51:45.091144 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.081351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45667 (local address 127.0.0.1:47366)
0812 14:51:45.081506 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.081543 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.082064 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.082359 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.082362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.082626 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.082862 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.082865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.083595 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.083597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.084717 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.084730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.085149 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.085150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.085325 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.085873 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.085885 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.087763 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.089739 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.089746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.089748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.089974 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.090199 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.090200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.090200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.090314 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.090620 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.090625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.090750 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.090839 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.090971 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260812 14:51:45.091833 23431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.146.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.188@KRBTEST.COM" created.
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:45.101451 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.092120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57932)
0812 14:51:45.092174 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.092186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.092231 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.092499 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.092500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.092646 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.092708 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.092709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.093628 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.093629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.094161 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.094163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.094648 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.094649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.094715 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.095173 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.095179 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.098049 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.100112 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.100114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.100114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.100372 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.100603 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.100604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.100604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.100678 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.100977 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.100978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.101032 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.101117 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.101272 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:51:45.101462 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.092189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.13.146.189:35851)
0812 14:51:45.092368 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.092372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.092388 (+ 16us) server_negotiation.cc:407] Connection header received
0812 14:51:45.092405 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.092406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.092416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.092498 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.092799 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.093546 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.094235 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.094559 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.094640 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.098145 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.098157 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.098157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.098165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.099998 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.100463 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.100466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.100466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.100522 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.100753 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.100753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.100753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.100907 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.100971 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.101155 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.101359 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260812 14:51:45.102188 23431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:45.102605 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.102450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45948)
0812 14:51:45.102509 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.102569 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 14:51:45.102783 23327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
I20260812 14:51:45.103214 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.103123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45962)
0812 14:51:45.103152 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.103183 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 14:51:45.103358 23431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:39229: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
I20260812 14:51:45.115658 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:39229
--webserver_interface=127.13.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:51:45.129076 23276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } attempt: 3
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:45.136561 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.129521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57934)
0812 14:51:45.129580 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.129600 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.129673 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.129853 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.129855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.130033 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.130109 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.130112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.130811 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.130812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.131356 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.131358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.131793 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.131794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.131872 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.132360 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.132366 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.133699 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.135548 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.135549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.135550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.135749 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.135967 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.135968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.135968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.136023 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.136306 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.136307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.136349 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.136402 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.136487 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:51:45.136626 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.129588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.13.146.189:35851)
0812 14:51:45.129656 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.129658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.129670 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:51:45.129761 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.129762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.129773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.129848 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.130196 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.130702 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.131449 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.131679 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.131790 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.133808 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.133810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.133810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.133819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.135459 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.135828 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.135829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.135829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.135897 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.136087 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.136088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.136088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.136241 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.136312 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.136434 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.136524 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:51:45.136993 23276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:45.137442 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.137321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45964)
0812 14:51:45.137379 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.137416 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 14:51:45.137795 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.137704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45976)
0812 14:51:45.137750 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.137777 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:51:45.137926 23276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:39229: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
I20260812 14:51:45.152622 23431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } attempt: 1
I20260812 14:51:45.153178 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.153024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45990)
0812 14:51:45.153091 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.153142 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:51:45.153587 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.153482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45992)
0812 14:51:45.153530 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.153561 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:51:45.153687 23431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:39229: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
I20260812 14:51:45.169094 23276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } attempt: 1
I20260812 14:51:45.169649 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.169507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:45996)
0812 14:51:45.169575 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.169616 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:51:45.170079 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.169971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46010)
0812 14:51:45.170021 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.170050 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:51:45.170228 23276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:39229: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
W20260812 14:51:45.204754 23442 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:45.204982 23442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:45.205000 23442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:45.205075 23442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:45.206813 23442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:45.206890 23442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:45.206913 23442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:45.206938 23442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:45.206964 23442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:45.208499 23442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:45667,127.13.146.189:35851,127.13.146.188:39229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:39229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:45.208792 23442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:45.209055 23442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:45.213966 23450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:45.213991 23449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:45.213987 23452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:45.214260 23442 server_base.cc:1034] running on GCE node
I20260812 14:51:45.214507 23442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:45.214761 23442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:45.215902 23442 hybrid_clock.cc:648] HybridClock initialized: now 1786546305215873 us; error 45 us; skew 500 ppm
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:45.218600 23442 init.cc:377] Logged in from keytab as kudu/127.13.146.188@KRBTEST.COM (short username kudu)
I20260812 14:51:45.219048 23442 webserver.cc:460] Webserver started at http://127.13.146.188:45375/ using document root <none> and password file <none>
I20260812 14:51:45.219290 23442 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:45.219362 23442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:45.219514 23442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:45.220594 23442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data/instance:
uuid: "9e6e076eb741458eae98a823b1700187"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "5c06de33b0ed7cfdac117bf5ef5c538a"
server_key_iv: "f4ac6038de0c725bffd10bf5257d9ba5"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.220983 23442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal/instance:
uuid: "9e6e076eb741458eae98a823b1700187"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "5c06de33b0ed7cfdac117bf5ef5c538a"
server_key_iv: "f4ac6038de0c725bffd10bf5257d9ba5"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.221822 23431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } attempt: 2
I20260812 14:51:45.222225 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.222082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46022)
0812 14:51:45.222144 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.222179 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:51:45.222537 23442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:45.222692 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.222591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46026)
0812 14:51:45.222643 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.222669 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 14:51:45.222807 23431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:39229: Network error: Client connection negotiation failed: client connection to 127.13.146.188:39229: connect: Connection refused (error 111)
I20260812 14:51:45.223452 23469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:45.223631 23442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:45.223757 23442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "9e6e076eb741458eae98a823b1700187"
format_stamp: "Formatted at 2026-08-12 14:51:45 on dist-test-slave-8hhm"
server_key: "5c06de33b0ed7cfdac117bf5ef5c538a"
server_key_iv: "f4ac6038de0c725bffd10bf5257d9ba5"
server_key_version: "encryptionkey@0"
I20260812 14:51:45.223837 23442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:45.237619 23442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:45.239688 23442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:45.239861 23442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:45.244117 23442 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.188:39229
I20260812 14:51:45.244154 23580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.188:39229 every 8 connection(s)
I20260812 14:51:45.244529 23442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
I20260812 14:51:45.245364 23276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } attempt: 2
I20260812 14:51:45.245579 23583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:45.247812 23583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:45.251864 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23442
I20260812 14:51:45.251965 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786546020513802-13898-0/minicluster-data/master-2/wal/instance
I20260812 14:51:45.252203 13898 external_mini_cluster.cc:1428] Setting key 762cf4199ac756d7863b51dfc57679a0
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:45.256109 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.245695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46030)
0812 14:51:45.245780 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.245792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.245843 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.246952 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.246953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.247138 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.247185 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.247187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.248500 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.248502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.249304 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.249307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.249913 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.249915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.249979 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.250501 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.250506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.252047 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.254252 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.254253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.254254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.254477 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.255079 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.255080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.255081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.255169 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.255684 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.255685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.255742 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.255807 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.256011 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 14:51:45.256536 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.245855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.13.146.188:39229)
0812 14:51:45.246055 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.246063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.246089 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:51:45.246746 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.246754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.246804 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.246969 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.247301 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.248384 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.249383 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.249836 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.249915 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.252145 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.252153 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.252153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.252164 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.254179 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.254567 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.254570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.254570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.254995 (+ 425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.255257 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.255259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.255259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.255574 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.255650 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.256017 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.256149 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:45.258850 23276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: Bootstrap starting.
I20260812 14:51:45.259696 23276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:45.260073 23276 log.cc:824] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:45.261278 23276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: No bootstrap required, opened a new log
I20260812 14:51:45.261816 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.248535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:45667 (local address 127.0.0.1:47378)
0812 14:51:45.248686 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.248724 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.248792 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.248950 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.248952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.249196 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.249373 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.249376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.249997 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.249998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.250742 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.250745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.251181 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.251182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.251270 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.252476 (+ 1206us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.252481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.254305 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.256193 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.256201 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.256203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.256475 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.256928 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.256929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.256929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.256997 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.261261 (+ 4264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.261271 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.261396 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.261520 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.261675 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260812 14:51:45.262277 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.248563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.13.146.190:45667)
0812 14:51:45.248628 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.248630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.248794 (+ 164us) server_negotiation.cc:407] Connection header received
0812 14:51:45.248850 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.248851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.248861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.248916 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.249454 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.249901 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.250825 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.251089 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.251146 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.254396 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.254398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.254398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.254405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.256105 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.256567 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.256568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.256568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.256627 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.257081 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.257082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.257083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.260418 (+ 3335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.260570 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.262090 (+ 1520us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.262181 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:51:45.262737 23583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:45.263204 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.252776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.13.146.188:39229)
0812 14:51:45.253264 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.253266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.253274 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:45.253297 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.253298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.253309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.253378 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.254179 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.254793 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.255842 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.256176 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.256247 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.259068 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.259070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.259070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.259077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.261436 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.261884 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.261886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.261886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.261969 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.262204 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.262205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.262205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.262339 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.262937 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.262984 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.263068 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":74,"threads_started":1}
I20260812 14:51:45.264029 23276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.264240 23276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22696f699dad4d73b4fff60f80e68e95, State: Initialized, Role: FOLLOWER
I20260812 14:51:45.264356 23276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.264676 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.264763 23276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:45.264853 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:51:45.266811 23611 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:45.266938 23611 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:45.270802 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.263025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:35851 (local address 127.0.0.1:57946)
0812 14:51:45.263068 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.263078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.263113 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.263233 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.263234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.263389 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.263451 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.263452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.264344 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.264345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.265136 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.265138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.265679 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.265681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.265766 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.266213 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.266218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.267743 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.269647 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.269648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.269648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.269866 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.270087 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.270088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.270088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.270153 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.270456 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.270457 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.270503 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.270570 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.270674 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:51:45.271164 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.263084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.13.146.189:35851)
0812 14:51:45.263129 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.263131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.263140 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:45.263166 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.263167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.263176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.263221 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.263539 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.264192 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.265251 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.265567 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.265625 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.267838 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.267840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.267840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.267848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.269555 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.269948 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.269950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.269950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.270015 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.270222 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.270223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.270223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.270363 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.270426 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.270994 (+ 568us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.271078 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:51:45.267933 23146 master_runner.cc:186] Error getting master registration for 127.13.146.189:35851: OK, instance_id { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" instance_seqno: 1786546305076190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:51:45.271380 23146 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:51:45.271471 23583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:45.272675 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.264217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.13.146.190:45667)
0812 14:51:45.264258 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.264260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.264267 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:51:45.264289 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.264289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.264297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.264345 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.264949 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.265663 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.266647 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.266936 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.266983 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.269767 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.269768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.269769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.269777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.271433 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.271886 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.271887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.271887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.271944 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.272152 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.272153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.272154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.272273 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.272458 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.272497 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.272575 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:45.278833 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.271706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46042)
0812 14:51:45.271842 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.271850 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.271890 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.272073 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.272074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.272224 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.272260 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.272261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.273139 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.273140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.273762 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.273765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.274248 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.274249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.274312 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.274735 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.274740 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.275950 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.277798 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.277799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.277799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.277970 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.278172 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.278172 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.278173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.278236 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.278558 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.278558 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.278592 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.278648 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.278740 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260812 14:51:45.279112 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.271790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.13.146.188:39229)
0812 14:51:45.271822 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.271824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.271937 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:51:45.271953 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.271953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.271961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.272017 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.272337 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.273036 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.273863 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.274160 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.274230 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.276049 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.276050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.276050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.276057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.277685 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.278043 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.278045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.278045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.278101 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.278296 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.278297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.278297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.278476 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.278536 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.278954 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.279029 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:51:45.281039 23583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: Bootstrap starting.
I20260812 14:51:45.281905 23583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:45.282261 23583 log.cc:824] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:45.282341 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.273274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.13.146.189:35851)
0812 14:51:45.273308 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.273309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.273316 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:51:45.273336 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.273336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.273345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.273402 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.273990 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.274632 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.276130 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.276428 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.276470 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.279208 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.279210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.279210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.279218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.280992 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.281420 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.281422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.281422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.281496 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.281740 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.281741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.281741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.281902 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.282117 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.282172 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.282250 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:51:45.283138 23583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: No bootstrap required, opened a new log
I20260812 14:51:45.283656 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:45.284682 23583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.284859 23583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e6e076eb741458eae98a823b1700187, State: Initialized, Role: FOLLOWER
I20260812 14:51:45.284927 23583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.284890 23228 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "22696f699dad4d73b4fff60f80e68e95"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47394
I20260812 14:51:45.285044 23228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:51:45.285090 23228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:45.285269 23583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:45.285202 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.285344 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:45.286063 23228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.286309 23228 leader_election.cc:288] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [CANDIDATE]: Term 1 election: Requested vote from peers ccdf6378f1914a338bc215e9dfa87e2c (127.13.146.189:35851), 9e6e076eb741458eae98a823b1700187 (127.13.146.188:39229)
I20260812 14:51:45.286897 23536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22696f699dad4d73b4fff60f80e68e95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9e6e076eb741458eae98a823b1700187"
I20260812 14:51:45.287006 23536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:45.287822 23536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22696f699dad4d73b4fff60f80e68e95 in term 1.
I20260812 14:51:45.288473 23178 leader_election.cc:302] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22696f699dad4d73b4fff60f80e68e95, 9e6e076eb741458eae98a823b1700187; no voters:
I20260812 14:51:45.288591 23595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:45.288733 23595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [term 1 LEADER]: Becoming Leader. State: Replica: 22696f699dad4d73b4fff60f80e68e95, State: Running, Role: LEADER
I20260812 14:51:45.288818 23595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.289197 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22696f699dad4d73b4fff60f80e68e95. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.289284 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:45.293335 23654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:45.291260 23442 master_runner.cc:186] Error getting master registration for 127.13.146.189:35851: OK, instance_id { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" instance_seqno: 1786546305076190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:51:45.293588 23442 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:51:45.293740 23654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 14:51:45.295483 23637 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:45.295562 23637 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:45.296641 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.287639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.13.146.190:45667)
0812 14:51:45.287681 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.287683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.287943 (+ 260us) server_negotiation.cc:407] Connection header received
0812 14:51:45.288054 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.288055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.288067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.288103 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.288928 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.289711 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.291010 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.291318 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.291360 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.293358 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.293360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.293360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.293368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.295250 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.295684 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.295685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.295685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.295746 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.295971 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.295972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.295972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.296113 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.296208 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.296453 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.296537 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:51:45.298228 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.287955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.13.146.189:35851)
0812 14:51:45.288024 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.288026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.288121 (+ 95us) server_negotiation.cc:407] Connection header received
0812 14:51:45.288269 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.288270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.288281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.288310 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.289026 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.289745 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.291011 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.291318 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.291360 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.294441 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.294443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.294443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.294451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.296218 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.296589 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.296590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.296590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.296657 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.297670 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.297671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.297672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.297835 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.297936 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.298050 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.298133 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 14:51:45.299101 23536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [term 1 FOLLOWER]: Refusing update from remote peer 22696f699dad4d73b4fff60f80e68e95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:45.299463 23651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:45.300848 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22696f699dad4d73b4fff60f80e68e95. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.300954 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:45.302561 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.288149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.13.146.188:39229)
0812 14:51:45.288233 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.288234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.290044 (+ 1810us) server_negotiation.cc:407] Connection header received
0812 14:51:45.290133 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.290134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.290145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.290174 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.292642 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.293505 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.297527 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.297818 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.297883 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.299427 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.299429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.299429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.299438 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.301101 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.301526 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.301527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.301527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.301583 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.301787 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.301788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.301788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.301913 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.301998 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.302394 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.302478 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 14:51:45.308961 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.309062 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:45.309160 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.309295 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:45.309460 23662 mvcc.cc:205] Tried to move back new op lower bound from 7317693666502279168 to 7317693666463555584. Current Snapshot: MvccSnapshot[applied={T|T < 7317693666502279168}]
I20260812 14:51:45.313920 23431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } attempt: 3
I20260812 14:51:45.317617 23654 catalog_manager.cc:1347] Generated new cluster ID: 7cdbd85b4b4c4f16bff8b638d3971589
I20260812 14:51:45.317677 23654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:51:45 dist-test-slave-8hhm krb5kdc[23116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546305, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:45.322381 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.314253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.13.146.188:39229)
0812 14:51:45.314320 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.314322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.314422 (+ 100us) server_negotiation.cc:407] Connection header received
0812 14:51:45.314444 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.314445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.314455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.314498 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.314864 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.315603 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.316610 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.316911 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.316983 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.319637 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:45.319638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:45.319639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:45.319646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:45.321188 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.321614 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.321615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.321616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.321671 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:45.321874 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:45.321875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:45.321875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:45.321992 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:45.322043 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:45.322217 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 14:51:45.322290 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:51:45.322383 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:45.314195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:39229 (local address 127.0.0.1:46058)
0812 14:51:45.314302 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:45.314315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:45.314368 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:45.314540 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:45.314542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:45.314734 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:45.314786 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.314788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.315681 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.315682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.316516 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:45.316518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:45.316983 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:45.316984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:45.317046 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:45.317599 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:45.317605 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:45.319548 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:45.321293 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.321294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.321295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.321531 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.321723 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:45.321724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:45.321725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:45.321805 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:45.322035 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:45.322036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:45.322094 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 14:51:45.322167 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:51:45.322286 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 14:51:45.324007 23431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: Bootstrap starting.
I20260812 14:51:45.324801 23431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:45.325104 23431 log.cc:824] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:45.326758 23654 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:45.327502 23654 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:45.337141 23431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: No bootstrap required, opened a new log
I20260812 14:51:45.339290 23431 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.339454 23431 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdf6378f1914a338bc215e9dfa87e2c, State: Initialized, Role: FOLLOWER
I20260812 14:51:45.339541 23431 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } }
I20260812 14:51:45.339888 23431 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:45.340008 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22696f699dad4d73b4fff60f80e68e95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ccdf6378f1914a338bc215e9dfa87e2c"
I20260812 14:51:45.340186 23379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:45.341116 23379 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22696f699dad4d73b4fff60f80e68e95 in term 1.
I20260812 14:51:45.341367 23376 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [term 1 FOLLOWER]: Refusing update from remote peer 22696f699dad4d73b4fff60f80e68e95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:45.341652 23651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:45.344615 23654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95: Generated new TSK 0
I20260812 14:51:45.344795 23654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:51:45.345309 23674 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.345414 23674 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:45.345465 23674 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22696f699dad4d73b4fff60f80e68e95. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.345506 23674 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:45.345534 23674 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.345589 23674 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:51:45.346060 23700 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:45.346109 23700 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:45.346565 23703 mvcc.cc:205] Tried to move back new op lower bound from 7317693666611744768 to 7317693666463555584. Current Snapshot: MvccSnapshot[applied={T|T < 7317693666502279168}]
I20260812 14:51:45.346977 23595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22696f699dad4d73b4fff60f80e68e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22696f699dad4d73b4fff60f80e68e95" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 45667 } } peers { permanent_uuid: "ccdf6378f1914a338bc215e9dfa87e2c" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 35851 } } peers { permanent_uuid: "9e6e076eb741458eae98a823b1700187" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 39229 } } }
I20260812 14:51:45.347074 23595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22696f699dad4d73b4fff60f80e68e95 [sys.catalog]: This master's current role is: LEADER
W20260812 14:51:45.374543 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.372773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.13.146.189:35851)
0812 14:51:45.372820 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.372822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.373813 (+ 991us) server_negotiation.cc:407] Connection header received
0812 14:51:45.373923 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.373924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.373935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.373967 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.374463 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57970: BlockingRecv error: recv got EOF from 127.0.0.1:57970 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 14:51:45.375329 23593 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.373157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.13.146.188:39229)
0812 14:51:45.373227 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.373229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.374798 (+ 1569us) server_negotiation.cc:407] Connection header received
0812 14:51:45.374897 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.374898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.374909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.374940 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.375268 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46062: BlockingRecv error: recv got EOF from 127.0.0.1:46062 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 14:51:45.376065 23284 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.373529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.13.146.190:45667)
0812 14:51:45.373579 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.373580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.375555 (+ 1975us) server_negotiation.cc:407] Connection header received
0812 14:51:45.375653 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.375654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.375664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.375693 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.376003 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47414: BlockingRecv error: recv got EOF from 127.0.0.1:47414 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 14:51:45.378517 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.377753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.13.146.189:35851)
0812 14:51:45.377788 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.377789 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.377979 (+ 190us) server_negotiation.cc:407] Connection header received
0812 14:51:45.378087 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.378088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.378107 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.378137 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.378469 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57974: BlockingRecv error: recv got EOF from 127.0.0.1:57974 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:51:45.381600 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.380819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.13.146.189:35851)
0812 14:51:45.380852 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.380853 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.381075 (+ 222us) server_negotiation.cc:407] Connection header received
0812 14:51:45.381171 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.381172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.381182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.381248 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.381558 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57988: BlockingRecv error: recv got EOF from 127.0.0.1:57988 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:51:45.385492 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.384739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.13.146.189:35851)
0812 14:51:45.384770 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.384771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.384977 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:51:45.385069 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.385070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.385079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.385107 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.385449 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57994: BlockingRecv error: recv got EOF from 127.0.0.1:57994 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:51:45.391506 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.390675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58008 (local address 127.13.146.189:35851)
0812 14:51:45.390722 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.390725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.390971 (+ 246us) server_negotiation.cc:407] Connection header received
0812 14:51:45.391069 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.391070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.391080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.391108 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.391453 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58008: BlockingRecv error: recv got EOF from 127.0.0.1:58008 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 14:51:45.401708 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.400779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.13.146.189:35851)
0812 14:51:45.400823 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.400824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.401073 (+ 249us) server_negotiation.cc:407] Connection header received
0812 14:51:45.401174 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.401175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.401186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.401276 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.401649 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58024: BlockingRecv error: recv got EOF from 127.0.0.1:58024 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:51:45.419958 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.419065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.13.146.189:35851)
0812 14:51:45.419109 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.419111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.419374 (+ 263us) server_negotiation.cc:407] Connection header received
0812 14:51:45.419479 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.419480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.419492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.419521 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.419900 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58034: BlockingRecv error: recv got EOF from 127.0.0.1:58034 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:51:45.454453 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.453455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.13.146.189:35851)
0812 14:51:45.453504 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.453506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.453802 (+ 296us) server_negotiation.cc:407] Connection header received
0812 14:51:45.453914 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.453915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.453928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.453959 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.454390 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58038: BlockingRecv error: recv got EOF from 127.0.0.1:58038 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 14:51:45.520968 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.520095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.13.146.189:35851)
0812 14:51:45.520188 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.520191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.520296 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:51:45.520394 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.520395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.520409 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.520477 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.520890 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58044: BlockingRecv error: recv got EOF from 127.0.0.1:58044 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 14:51:45.651021 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.650239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.13.146.189:35851)
0812 14:51:45.650333 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.650335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.650447 (+ 112us) server_negotiation.cc:407] Connection header received
0812 14:51:45.650530 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.650531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.650544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.650598 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.650952 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58048: BlockingRecv error: recv got EOF from 127.0.0.1:58048 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 14:51:45.908957 23435 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:45.908218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.13.146.189:35851)
0812 14:51:45.908275 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:45.908279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:45.908316 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:51:45.908428 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:45.908429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:45.908441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:45.908520 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:45.908889 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58064: BlockingRecv error: recv got EOF from 127.0.0.1:58064 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:51:46.296120 23637 catalog_manager.cc:1259] Loaded cluster ID: 7cdbd85b4b4c4f16bff8b638d3971589
I20260812 14:51:46.296211 23637 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: loading cluster ID for follower catalog manager: success
I20260812 14:51:46.297065 23637 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: acquiring CA information for follower catalog manager: success
I20260812 14:51:46.297436 23637 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9e6e076eb741458eae98a823b1700187: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:51:46.346532 23700 catalog_manager.cc:1259] Loaded cluster ID: 7cdbd85b4b4c4f16bff8b638d3971589
I20260812 14:51:46.346618 23700 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: loading cluster ID for follower catalog manager: success
I20260812 14:51:46.347409 23700 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: acquiring CA information for follower catalog manager: success
I20260812 14:51:46.347806 23700 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccdf6378f1914a338bc215e9dfa87e2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:51:46.425108 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:46.422283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.13.146.189:35851)
0812 14:51:46.422426 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:46.422428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:46.422441 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:51:46.422531 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:46.422531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:46.422606 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:46.422702 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:46.422860 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.423405 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.424230 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.424439 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.424518 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:46.424729 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:46.424845 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:46.424941 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:46.424964 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 14:51:46.424989 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260812 14:51:46.429575 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:46.427036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.13.146.188:39229)
0812 14:51:46.427175 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:46.427177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:46.427186 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:46.427229 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:46.427230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:46.427269 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:46.427339 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:46.427430 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.427993 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.428743 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.428939 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.428995 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:46.429139 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:46.429310 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:46.429372 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:46.429407 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:51:46.429426 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260812 14:51:46.434262 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:46.431630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.13.146.188:39229)
0812 14:51:46.431697 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:46.431699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:46.431870 (+ 171us) server_negotiation.cc:407] Connection header received
0812 14:51:46.431949 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:46.431951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:46.431967 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:46.432019 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:46.432243 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.432709 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.433566 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.433759 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.433832 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:46.433983 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:46.434021 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:46.434075 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:46.434144 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 14:51:46.434167 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:51:46.434875 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:46.431378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.13.146.189:35851)
0812 14:51:46.431426 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:46.431428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:46.431889 (+ 461us) server_negotiation.cc:407] Connection header received
0812 14:51:46.432012 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:46.432013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:46.432027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:46.432083 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:46.432281 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.432996 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.434012 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.434301 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.434339 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:46.434539 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:46.434571 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:46.434597 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:46.434749 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:51:46.434783 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:51:46.434979 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:46.431397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.13.146.190:45667)
0812 14:51:46.431527 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:46.431529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:46.431547 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:51:46.431586 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:46.431587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:46.431624 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:46.431751 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:46.431928 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.432743 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.433887 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:46.434193 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:46.434262 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:46.434501 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:46.434617 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:46.434677 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:46.434782 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:51:46.434895 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260812 14:51:46.436695 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23146
I20260812 14:51:46.454542 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23290
I20260812 14:51:46.467960 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23442
2026-08-12T14:51:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3138 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:51:46 dist-test-slave-8hhm krb5kdc[23858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:51:46 dist-test-slave-8hhm krb5kdc[23858](info): set up 2 sockets
Aug 12 14:51:46 dist-test-slave-8hhm krb5kdc[23858](info): commencing operation
krb5kdc: starting...
W20260812 14:51:47.891410 13898 mini_kdc.cc:121] Time spent starting KDC: real 1.398s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/test-user.keytab.
Aug 12 14:51:47 dist-test-slave-8hhm krb5kdc[23858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.146.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
Entry for principal kudu/127.13.146.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab.
I20260812 14:51:48.020345 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46679
--webserver_interface=127.13.146.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:51:48.105089 23894 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:48.105436 23894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:48.105473 23894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:48.105517 23894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:48.106722 23894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:48.106805 23894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:48.106829 23894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:48.106851 23894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:48.106882 23894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:48.108402 23894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.190.keytab
--principal=kudu/127.13.146.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.190:46679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.146.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:48.108700 23894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:48.108938 23894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:48.114110 23902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:48.114264 23894 server_base.cc:1034] running on GCE node
W20260812 14:51:48.114131 23900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:48.114372 23906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:48.114648 23894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:48.115005 23894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:48.116171 23894 hybrid_clock.cc:648] HybridClock initialized: now 1786546308116137 us; error 47 us; skew 500 ppm
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:48.118834 23894 init.cc:377] Logged in from keytab as kudu/127.13.146.190@KRBTEST.COM (short username kudu)
I20260812 14:51:48.119288 23894 webserver.cc:460] Webserver started at http://127.13.146.190:45881/ using document root <none> and password file <none>
I20260812 14:51:48.119520 23894 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:48.119566 23894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:48.119719 23894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:48.120826 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data/instance:
uuid: "afa0611492be487cb60568801d1e42f3"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "c353b4467848cd70d1b1b6540b06b046"
server_key_iv: "c87fb0e0a8f8f9f3c2ec4a62c0cf8b1f"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.121255 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance:
uuid: "afa0611492be487cb60568801d1e42f3"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "c353b4467848cd70d1b1b6540b06b046"
server_key_iv: "c87fb0e0a8f8f9f3c2ec4a62c0cf8b1f"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.122676 23894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:48.123478 23922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:48.123692 23894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:48.123804 23894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal
uuid: "afa0611492be487cb60568801d1e42f3"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "c353b4467848cd70d1b1b6540b06b046"
server_key_iv: "c87fb0e0a8f8f9f3c2ec4a62c0cf8b1f"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.123888 23894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:48.135582 23894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:48.137585 23894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:48.137759 23894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:48.141935 23894 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.190:46679
I20260812 14:51:48.142004 24017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.190:46679 every 8 connection(s)
I20260812 14:51:48.142377 23894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/data/info.pb
I20260812 14:51:48.143342 24019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:48.145051 24019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:48.146139 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 23894
I20260812 14:51:48.146261 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-0/wal/instance
I20260812 14:51:48.146466 13898 external_mini_cluster.cc:1428] Setting key e9799e6c5262e75afb9b9c7e212c9a6c
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.189@KRBTEST.COM; defaulting to no policy
I20260812 14:51:48.156555 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.145779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46679 (local address 127.0.0.1:41290)
0812 14:51:48.146017 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.146062 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.146701 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.146999 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.147002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.147331 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.147487 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.147492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.148897 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.148898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.149857 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.149860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.150369 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.150371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.150464 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.151037 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.151047 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.152746 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.155009 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.155016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.155018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.155227 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.155513 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.155514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.155514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.155598 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.156051 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.156056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.156113 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.156234 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.156383 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260812 14:51:48.156566 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.145843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.13.146.190:46679)
0812 14:51:48.146028 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.146038 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.146168 (+ 130us) server_negotiation.cc:407] Connection header received
0812 14:51:48.146866 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.146873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.146892 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.146992 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.147609 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.148785 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.149958 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.150273 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.150355 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.152841 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.152852 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.152852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.152861 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.154909 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.155315 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.155319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.155319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.155433 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.155669 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.155670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.155670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.155977 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.156072 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.156291 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.156470 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260812 14:51:48.157464 24019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:48.157869 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.157756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56422)
0812 14:51:48.157803 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.157849 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:51:48.158046 23924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
I20260812 14:51:48.158469 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.158384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56438)
0812 14:51:48.158427 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.158452 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:51:48.158665 24019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:38125: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
Entry for principal kudu/127.13.146.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab.
I20260812 14:51:48.176065 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:38125
--webserver_interface=127.13.146.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:51:48.207980 24019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } attempt: 1
I20260812 14:51:48.208580 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.208435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56440)
0812 14:51:48.208504 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.208550 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 14:51:48.208979 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.208888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56446)
0812 14:51:48.208934 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.208960 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:51:48.209108 24019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:38125: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
W20260812 14:51:48.263819 24030 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:48.264053 24030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:48.264071 24030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:48.264093 24030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:48.265205 24030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:48.265331 24030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:48.265347 24030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:48.265360 24030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:48.265373 24030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:48.266937 24030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.189.keytab
--principal=kudu/127.13.146.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.189:38125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.146.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:48.267145 24030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:48.267350 24030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:51:48.272572 24043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:48.272584 24046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:48.272584 24041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:48.272884 24030 server_base.cc:1034] running on GCE node
I20260812 14:51:48.273191 24030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:48.273571 24030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:48.274766 24030 hybrid_clock.cc:648] HybridClock initialized: now 1786546308274751 us; error 45 us; skew 500 ppm
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:48.277276 24019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } attempt: 2
I20260812 14:51:48.277534 24030 init.cc:377] Logged in from keytab as kudu/127.13.146.189@KRBTEST.COM (short username kudu)
I20260812 14:51:48.277697 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.277559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56452)
0812 14:51:48.277622 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.277657 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:51:48.278053 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.277944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56454)
0812 14:51:48.277991 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.278024 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:51:48.278100 24030 webserver.cc:460] Webserver started at http://127.13.146.189:33951/ using document root <none> and password file <none>
W20260812 14:51:48.278162 24019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.189:38125: Network error: Client connection negotiation failed: client connection to 127.13.146.189:38125: connect: Connection refused (error 111)
I20260812 14:51:48.278373 24030 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:48.278445 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:48.278574 24030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:48.279870 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data/instance:
uuid: "495454a403b841d292108ecea46ed6e8"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "94d2cffecf36b10ff0a0fa5ba7a5fb3e"
server_key_iv: "d5293598695ce2bc3047f7cc00abc3e1"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.280265 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal/instance:
uuid: "495454a403b841d292108ecea46ed6e8"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "94d2cffecf36b10ff0a0fa5ba7a5fb3e"
server_key_iv: "d5293598695ce2bc3047f7cc00abc3e1"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.281759 24030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:51:48.282541 24055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:48.282734 24030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:48.282837 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal
uuid: "495454a403b841d292108ecea46ed6e8"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "94d2cffecf36b10ff0a0fa5ba7a5fb3e"
server_key_iv: "d5293598695ce2bc3047f7cc00abc3e1"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.282913 24030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:48.294032 24030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:48.295931 24030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:48.296096 24030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:48.300525 24030 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.189:38125
I20260812 14:51:48.300570 24170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.189:38125 every 8 connection(s)
I20260812 14:51:48.300966 24030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/data/info.pb
I20260812 14:51:48.301937 24172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:48.302013 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24030
I20260812 14:51:48.302122 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-1/wal/instance
I20260812 14:51:48.302356 13898 external_mini_cluster.cc:1428] Setting key bef8e5d4e51c9b25da8ad0718d8fd114
I20260812 14:51:48.303675 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.146.188@KRBTEST.COM; defaulting to no policy
I20260812 14:51:48.313911 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.304459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.13.146.190:46679)
0812 14:51:48.304536 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.304539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.304730 (+ 191us) server_negotiation.cc:407] Connection header received
0812 14:51:48.305200 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.305202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.305250 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.305329 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.305961 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.306756 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.307789 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.308152 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.308253 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.310970 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.310971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.310972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.310979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.312540 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.312961 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.312962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.312963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.313021 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.313313 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.313314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.313314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.313427 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.313482 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.313756 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.313829 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:51:48.314033 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.304381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46679 (local address 127.0.0.1:41292)
0812 14:51:48.304595 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.304641 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.305107 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.305394 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.305397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.305655 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.305859 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.305863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.306887 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.306889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.307692 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.307694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.308273 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.308274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.308391 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.309008 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.309017 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.310851 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.312654 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.312661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.312663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.312890 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.313084 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.313085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.313085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.313190 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.313489 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.313495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.313621 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.313707 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.313835 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.146.188@KRBTEST.COM" created.
I20260812 14:51:48.314639 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:48.322474 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.314921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56468)
0812 14:51:48.314970 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.314981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.315024 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.315357 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.315358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.315474 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.315524 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.315525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.316705 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.316706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.317270 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.317272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.317742 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.317743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.317810 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.318228 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.318233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.319456 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.321360 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.321361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.321361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.321535 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.321768 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.321769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.321770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.321826 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.322132 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.322133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.322172 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.322232 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.322339 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:51:48.322510 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.315009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.13.146.189:38125)
0812 14:51:48.315184 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.315189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.315213 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:51:48.315246 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.315249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.315264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.315343 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.315598 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.316616 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.317337 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.317644 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.317738 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.319543 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.319552 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.319552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.319559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.321205 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.321633 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.321636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.321637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.321697 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.321899 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.321900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.321900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.322050 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.322147 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.322278 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.322395 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260812 14:51:48.323138 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:48.323536 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.323421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43080)
0812 14:51:48.323470 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.323516 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:51:48.323678 24056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
I20260812 14:51:48.324116 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.324031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43096)
0812 14:51:48.324073 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.324099 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:51:48.324316 24172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:34297: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
Entry for principal kudu/127.13.146.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab.
I20260812 14:51:48.330284 13898 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskS5QCDN/build/release/bin/kudu
/tmp/dist-test-taskS5QCDN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:34297
--webserver_interface=127.13.146.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.146.148:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:51:48.369318 24019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } attempt: 3
I20260812 14:51:48.373535 24172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } attempt: 1
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:48.373978 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.373827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43108)
0812 14:51:48.373911 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.373940 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 14:51:48.374356 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.374264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43122)
0812 14:51:48.374303 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.374329 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:51:48.374454 24172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:34297: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
I20260812 14:51:48.376937 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.369845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.13.146.189:38125)
0812 14:51:48.369932 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.369934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.369945 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:51:48.369980 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.369981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.369990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.370050 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.370430 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.370944 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.371681 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.371928 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.372016 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.374269 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.374270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.374271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.374279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.375821 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.376259 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.376260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.376260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.376314 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.376508 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.376509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.376509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.376612 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.376665 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.376826 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.376883 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 14:51:48.377022 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.369741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56478)
0812 14:51:48.369808 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.369822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.369894 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.370070 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.370071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.370266 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.370333 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.370335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.371049 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.371051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.371580 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.371582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.372013 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.372014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.372104 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.372651 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.372656 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.374171 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.375923 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.375925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.375925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.376172 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.376378 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.376378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.376379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.376449 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.376677 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.376678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.376732 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.376810 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.376914 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:51:48.377442 24019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:48.377810 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.377692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43132)
0812 14:51:48.377744 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.377782 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:51:48.378144 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.378043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43140)
0812 14:51:48.378096 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.378125 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 14:51:48.378252 24019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:34297: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
I20260812 14:51:48.409428 24019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } attempt: 1
I20260812 14:51:48.410034 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.409856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43144)
0812 14:51:48.409952 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.409998 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 14:51:48.410506 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.410404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43146)
0812 14:51:48.410453 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.410480 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 14:51:48.410640 24019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:34297: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
W20260812 14:51:48.418525 24181 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:51:48.418749 24181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:51:48.418768 24181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:51:48.418818 24181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:51:48.420024 24181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:51:48.420066 24181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:51:48.420080 24181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:51:48.420094 24181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:51:48.420105 24181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:51:48.421736 24181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.146.148:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.146.190:46679,127.13.146.189:38125,127.13.146.188:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/krb5kdc/kudu_127.13.146.188.keytab
--principal=kudu/127.13.146.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.146.188:34297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.146.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:51:48.422014 24181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:51:48.422266 24181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:51:48.427981 24181 server_base.cc:1034] running on GCE node
W20260812 14:51:48.428025 24191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:48.428027 24188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:51:48.428289 24194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:51:48.428598 24181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:51:48.428925 24181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:51:48.430073 24181 hybrid_clock.cc:648] HybridClock initialized: now 1786546308430054 us; error 35 us; skew 500 ppm
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:51:48.432637 24181 init.cc:377] Logged in from keytab as kudu/127.13.146.188@KRBTEST.COM (short username kudu)
I20260812 14:51:48.433095 24181 webserver.cc:460] Webserver started at http://127.13.146.188:39959/ using document root <none> and password file <none>
I20260812 14:51:48.433393 24181 fs_manager.cc:359] Metadata directory not provided
I20260812 14:51:48.433444 24181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:51:48.433599 24181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:51:48.434751 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data/instance:
uuid: "1ed3624697bc434f9b717f281d285958"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "ef63a23e70f09aa85199e414e54ec3b7"
server_key_iv: "bde71f76c3f8ecd5c0916efa13d89916"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.435149 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal/instance:
uuid: "1ed3624697bc434f9b717f281d285958"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "ef63a23e70f09aa85199e414e54ec3b7"
server_key_iv: "bde71f76c3f8ecd5c0916efa13d89916"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.436596 24181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:51:48.437402 24209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:51:48.437582 24181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:51:48.437711 24181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data,/tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal
uuid: "1ed3624697bc434f9b717f281d285958"
format_stamp: "Formatted at 2026-08-12 14:51:48 on dist-test-slave-8hhm"
server_key: "ef63a23e70f09aa85199e414e54ec3b7"
server_key_iv: "bde71f76c3f8ecd5c0916efa13d89916"
server_key_version: "encryptionkey@0"
I20260812 14:51:48.437793 24181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:51:48.442597 24172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } attempt: 2
I20260812 14:51:48.443040 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.442901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43156)
0812 14:51:48.442965 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.443008 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:51:48.443429 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.443348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43160)
0812 14:51:48.443387 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.443414 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 14:51:48.443550 24172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.146.188:34297: Network error: Client connection negotiation failed: client connection to 127.13.146.188:34297: connect: Connection refused (error 111)
I20260812 14:51:48.457713 24181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:51:48.459707 24181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:51:48.459825 24181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:51:48.463847 24181 rpc_server.cc:307] RPC server started. Bound to: 127.13.146.188:34297
I20260812 14:51:48.463928 24316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.146.188:34297 every 8 connection(s)
I20260812 14:51:48.464293 24181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/data/info.pb
I20260812 14:51:48.465344 24317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:51:48.466291 13898 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskS5QCDN/build/release/bin/kudu as pid 24181
I20260812 14:51:48.466385 13898 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskS5QCDN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786546020513802-13898-0/minicluster-data/master-2/wal/instance
I20260812 14:51:48.466629 13898 external_mini_cluster.cc:1428] Setting key c54988145adab0827bb3ce3ecf64e99d
I20260812 14:51:48.467258 24317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:48.477553 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.468015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.13.146.190:46679)
0812 14:51:48.468065 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.468066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.468214 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:51:48.468296 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.468298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.468308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.468366 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.469164 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.469949 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.471154 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.471420 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.471492 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.474061 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.474063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.474063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.474078 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.475849 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.476335 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.476336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.476337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.476405 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.476752 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.476753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.476753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.476929 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.476997 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.477356 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.477444 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:51:48.477982 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.467931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.190:46679 (local address 127.0.0.1:41300)
0812 14:51:48.468083 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.468125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.468237 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.468402 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.468404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.468818 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.469065 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.469072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.470030 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.470031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.471061 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.471064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.471508 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.471509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.471610 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.472165 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.472194 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.473953 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.475944 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.475951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.475953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.476254 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.476505 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.476505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.476506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.476657 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.477006 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.477014 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.477177 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.477325 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.477520 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260812 14:51:48.477983 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.467195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.13.146.188:34297)
0812 14:51:48.467414 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.467423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.467461 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:51:48.467999 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.468009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.468130 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.468276 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.469094 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.470259 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.471287 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.471712 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.471778 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.474587 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.474594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.474595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.474616 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.476433 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.476807 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.476810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.476811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.476864 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.477094 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.477095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.477095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.477285 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.477448 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.477532 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.477611 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260812 14:51:48.478782 24317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.190@KRBTEST.COM
I20260812 14:51:48.485832 24019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } attempt: 2
I20260812 14:51:48.486987 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.479210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.13.146.189:38125)
0812 14:51:48.479266 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.479268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.479277 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:48.479303 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.479304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.479315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.479348 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.479761 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.480478 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.481472 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.481801 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.481864 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.483914 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.483915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.483916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.483923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.485591 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.486074 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.486076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.486076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.486146 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.486369 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.486370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.486370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.486571 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.486643 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.486846 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.486923 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:51:48.487322 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.479095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.189:38125 (local address 127.0.0.1:56482)
0812 14:51:48.479147 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.479158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.479193 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.479420 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.479420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.479585 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.479648 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.479650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.480567 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.480568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.481375 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.481377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.481874 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.481875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.481941 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.482446 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.482452 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.483824 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.485703 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.485705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.485705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.485948 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.486244 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.486245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.486246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.486317 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.486673 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.486674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.486723 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.487126 (+ 403us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.487229 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:51:48.487756 24317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:51:48.488372 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.478907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.13.146.190:46679)
0812 14:51:48.478942 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.478944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.479036 (+ 92us) server_negotiation.cc:407] Connection header received
0812 14:51:48.479093 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.479094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.479106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.479162 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.479752 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.480479 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.481471 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.481764 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.481837 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.484624 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.484626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.484627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.484634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.486698 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.487179 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.487181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.487181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.487241 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.487421 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.487422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.487422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.487537 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.487715 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.488190 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.488275 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.190@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.188@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:48.493713 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.486109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43186)
0812 14:51:48.486221 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.486233 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.486271 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.486472 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.486473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.486719 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.486768 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.486770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.487664 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.487665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.488471 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.488475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.488980 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.488981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.489045 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.489450 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.489454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.490573 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.492478 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.492479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.492480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.492697 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.492972 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.492973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.492973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.493040 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.493378 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.493380 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.493436 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.493493 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.493591 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 14:51:48.494068 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.486236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.13.146.188:34297)
0812 14:51:48.486281 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.486282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.486292 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:48.486315 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.486316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.486325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.486446 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.486836 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.487581 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.488541 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.488898 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.488980 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.490664 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.490665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.490665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.490672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.492375 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.492834 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.492835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.492836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.492891 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.493117 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.493117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.493118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.493282 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.493850 (+ 568us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.493902 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.493973 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":24}
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.146.189@KRBTEST.COM
I20260812 14:51:48.496275 24019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: Bootstrap starting.
I20260812 14:51:48.496706 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.487999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43190)
0812 14:51:48.488035 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.488044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.488085 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.488375 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.488377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.488530 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.488562 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.488564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.489502 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.489503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.490441 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.490445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.490980 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.490981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.491045 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.491373 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.491379 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.492901 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.495388 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.495389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.495389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.495598 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.495789 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.495789 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.495790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.495846 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.496354 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.496355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.496397 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.496489 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.496596 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:51:48.497007 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.488081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.13.146.188:34297)
0812 14:51:48.488260 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.488261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.488267 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:51:48.488283 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.488284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.488291 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.488349 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.488629 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.489413 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.490574 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.490902 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.490980 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.492976 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.492977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.492977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.492983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.495293 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.495666 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.495666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.495667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.495720 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.496127 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.496128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.496128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.496275 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.496362 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.496512 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.496913 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":327,"server-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260812 14:51:48.497293 24019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:48.497686 24019 log.cc:824] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:48.498672 24019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: No bootstrap required, opened a new log
I20260812 14:51:48.498778 24317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: Bootstrap starting.
I20260812 14:51:48.499573 24317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:48.499946 24317 log.cc:824] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:48.500702 24317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: No bootstrap required, opened a new log
I20260812 14:51:48.501413 24019 raft_consensus.cc:348] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.501603 24019 raft_consensus.cc:729] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa0611492be487cb60568801d1e42f3, State: Initialized, Role: FOLLOWER
I20260812 14:51:48.501713 24019 consensus_queue.cc:260] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.502094 24332 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.502166 24019 sys_catalog.cc:564] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:48.502220 24332 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:48.503006 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.488992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.13.146.189:38125)
0812 14:51:48.489024 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.489026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.489115 (+ 89us) server_negotiation.cc:407] Connection header received
0812 14:51:48.489143 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.489144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.489157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.489239 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.490058 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.490917 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.492041 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.492279 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.492385 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.495630 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.495632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.495632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.495641 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.501103 (+ 5462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.501618 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.501620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.501621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.501701 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.502005 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.502007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.502007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.502153 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.502427 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.502836 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.502922 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:51:48.502924 24317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.503064 24317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed3624697bc434f9b717f281d285958, State: Initialized, Role: FOLLOWER
I20260812 14:51:48.503151 24317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.503423 24346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.503499 24317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:48.503578 24346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:51:48.504570 24356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:48.504673 24356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:48.505702 13898 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:51:48.505649 23894 master_runner.cc:186] Error getting master registration for 127.13.146.189:38125: OK, instance_id { permanent_uuid: "495454a403b841d292108ecea46ed6e8" instance_seqno: 1786546308299315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:51:48.508461 23894 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:51:48.508556 23959 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "afa0611492be487cb60568801d1e42f3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41308
I20260812 14:51:48.508680 23959 raft_consensus.cc:482] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:51:48.508738 23959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:48.509765 23959 raft_consensus.cc:504] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.510058 23959 leader_election.cc:288] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [CANDIDATE]: Term 1 election: Requested vote from peers 495454a403b841d292108ecea46ed6e8 (127.13.146.189:38125), 1ed3624697bc434f9b717f281d285958 (127.13.146.188:34297)
W20260812 14:51:48.510203 24378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:48.510329 24378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:48.510653 24271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afa0611492be487cb60568801d1e42f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1ed3624697bc434f9b717f281d285958"
I20260812 14:51:48.510766 24271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:48.508076 24181 master_runner.cc:186] Error getting master registration for 127.13.146.189:38125: OK, instance_id { permanent_uuid: "495454a403b841d292108ecea46ed6e8" instance_seqno: 1786546308299315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:51:48.511329 24181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:51:48.511670 24271 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa0611492be487cb60568801d1e42f3 in term 1.
I20260812 14:51:48.511885 23924 leader_election.cc:302] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ed3624697bc434f9b717f281d285958, afa0611492be487cb60568801d1e42f3; no voters:
I20260812 14:51:48.512036 24332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:51:48.512261 24332 raft_consensus.cc:686] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [term 1 LEADER]: Becoming Leader. State: Replica: afa0611492be487cb60568801d1e42f3, State: Running, Role: LEADER
I20260812 14:51:48.512374 24332 consensus_queue.cc:237] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.512898 24381 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa0611492be487cb60568801d1e42f3. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.513059 24381 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:48.518146 24403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:51:48.518396 24403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:51:48.525708 24271 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [term 1 FOLLOWER]: Refusing update from remote peer afa0611492be487cb60568801d1e42f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:48.526103 24381 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:48.527626 24346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa0611492be487cb60568801d1e42f3. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.527774 24346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:48.529640 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.517664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.13.146.190:46679)
0812 14:51:48.517715 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.517718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.517916 (+ 198us) server_negotiation.cc:407] Connection header received
0812 14:51:48.518100 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.518101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.518112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.518146 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.519745 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.520557 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.522058 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.522357 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.522400 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.524981 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.524983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.524983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.524991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.527446 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.528389 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.528404 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.528404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.528468 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.528813 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.528815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.528815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.528961 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.529076 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.529463 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.529546 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:51:48.534757 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.517960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.13.146.189:38125)
0812 14:51:48.518024 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.518025 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.519711 (+ 1686us) server_negotiation.cc:407] Connection header received
0812 14:51:48.519827 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.519829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.519841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.519870 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.520564 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.521311 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.523836 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.524843 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.524887 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.530974 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.530976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.530977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.530985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.532735 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.533386 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.533401 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.533401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.533464 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.533849 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.533850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.533851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.534031 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.534130 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.534553 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.534646 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:51:48.534854 24172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } attempt: 3
I20260812 14:51:48.535784 24332 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.535904 24332 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:48.535986 24408 mvcc.cc:205] Tried to move back new op lower bound from 7317693679708094464 to 7317693679667531776. Current Snapshot: MvccSnapshot[applied={T|T < 7317693679708094464}]
I20260812 14:51:48.536159 24346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.536266 24346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:48.536684 24413 mvcc.cc:205] Tried to move back new op lower bound from 7317693679708094464 to 7317693679667531776. Current Snapshot: MvccSnapshot[applied={T|T < 7317693679708094464}]
Aug 12 14:51:48 dist-test-slave-8hhm krb5kdc[23858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546308, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.146.189@KRBTEST.COM for kudu/127.13.146.188@KRBTEST.COM
I20260812 14:51:48.542249 24403 catalog_manager.cc:1347] Generated new cluster ID: 559dffc4cdc7424b95e3afa023f6a09f
I20260812 14:51:48.542608 24403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:51:48.542963 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.519562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.13.146.188:34297)
0812 14:51:48.519623 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.519625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.522909 (+ 3284us) server_negotiation.cc:407] Connection header received
0812 14:51:48.523008 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.523009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.523022 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.523052 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.529081 (+ 6029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.529892 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.530938 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.531591 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.531660 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.538368 (+ 6708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.538369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.538370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.538378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.540212 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.541296 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.541298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.541298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.541369 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.542296 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.542297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.542297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.542446 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.542542 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.542781 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.542875 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:51:48.547216 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.535117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.146.188:34297 (local address 127.0.0.1:43206)
0812 14:51:48.535281 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:51:48.535292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:51:48.535333 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:51:48.536339 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:51:48.536340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:51:48.536500 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:51:48.536565 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.536567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.537395 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.537396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.538210 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:51:48.538213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.539273 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.539275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:51:48.539340 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.540310 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:51:48.540316 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:51:48.541957 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:51:48.546089 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.546090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.546090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.546313 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.546544 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:51:48.546545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:51:48.546545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:51:48.546610 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:51:48.546913 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:51:48.546914 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:51:48.546969 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:51:48.547027 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:51:48.547122 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260812 14:51:48.547897 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:48.535571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.13.146.188:34297)
0812 14:51:48.535704 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.535706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.535713 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:51:48.535743 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.535744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.535753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.535807 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.536644 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.537338 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.538296 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:48.538578 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:48.538642 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:48.542049 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:51:48.542051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:51:48.542051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:51:48.542058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:51:48.545196 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.546400 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.546401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.546401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.546469 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:51:48.546681 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:51:48.546682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:51:48.546682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:51:48.546824 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:51:48.546876 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:48.547737 (+ 861us) server_negotiation.cc:299] Negotiation successful
0812 14:51:48.547814 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 14:51:48.549476 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: Bootstrap starting.
I20260812 14:51:48.550264 24172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: Neither blocks nor log segments found. Creating new log.
I20260812 14:51:48.550585 24172 log.cc:824] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:51:48.551527 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: No bootstrap required, opened a new log
I20260812 14:51:48.555575 24172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.555913 24172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495454a403b841d292108ecea46ed6e8, State: Initialized, Role: FOLLOWER
I20260812 14:51:48.556111 24172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } }
I20260812 14:51:48.556677 24113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afa0611492be487cb60568801d1e42f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "495454a403b841d292108ecea46ed6e8"
I20260812 14:51:48.556775 24113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:51:48.557579 24113 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa0611492be487cb60568801d1e42f3 in term 1.
I20260812 14:51:48.557955 24110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [term 1 FOLLOWER]: Refusing update from remote peer afa0611492be487cb60568801d1e42f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:51:48.558214 24172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:51:48.558225 24332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:51:48.560257 24437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.560413 24437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:48.560508 24437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa0611492be487cb60568801d1e42f3. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.560591 24437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:51:48.560650 24437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.560734 24437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:51:48.563889 24455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:51:48.563990 24455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:51:48.586658 24332 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afa0611492be487cb60568801d1e42f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afa0611492be487cb60568801d1e42f3" member_type: VOTER last_known_addr { host: "127.13.146.190" port: 46679 } } peers { permanent_uuid: "495454a403b841d292108ecea46ed6e8" member_type: VOTER last_known_addr { host: "127.13.146.189" port: 38125 } } peers { permanent_uuid: "1ed3624697bc434f9b717f281d285958" member_type: VOTER last_known_addr { host: "127.13.146.188" port: 34297 } } }
I20260812 14:51:48.586797 24332 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:51:48.589157 24403 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:51:48.589955 24403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:51:48.597906 24403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afa0611492be487cb60568801d1e42f3: Generated new TSK 0
I20260812 14:51:48.598114 24403 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:51:48.631078 24024 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.629436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.13.146.190:46679)
0812 14:51:48.629481 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.629483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.630367 (+ 884us) server_negotiation.cc:407] Connection header received
0812 14:51:48.630474 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.630475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.630486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.630518 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.630984 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41332: BlockingRecv error: recv got EOF from 127.0.0.1:41332 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:51:48.631906 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.629824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.13.146.189:38125)
0812 14:51:48.629864 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.629866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.631329 (+ 1463us) server_negotiation.cc:407] Connection header received
0812 14:51:48.631431 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.631432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.631444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.631473 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.631842 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56492: BlockingRecv error: recv got EOF from 127.0.0.1:56492 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:51:48.632670 24325 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.630132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.13.146.188:34297)
0812 14:51:48.630176 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.630178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.632172 (+ 1994us) server_negotiation.cc:407] Connection header received
0812 14:51:48.632269 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.632270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.632281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.632311 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.632609 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43216: BlockingRecv error: recv got EOF from 127.0.0.1:43216 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:51:48.634969 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.634267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.13.146.189:38125)
0812 14:51:48.634314 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.634316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.634470 (+ 154us) server_negotiation.cc:407] Connection header received
0812 14:51:48.634576 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.634577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.634595 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.634624 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.634922 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56500: BlockingRecv error: recv got EOF from 127.0.0.1:56500 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:51:48.638019 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.637266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.13.146.189:38125)
0812 14:51:48.637309 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.637320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.637534 (+ 214us) server_negotiation.cc:407] Connection header received
0812 14:51:48.637616 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.637617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.637626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.637651 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.637957 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56512: BlockingRecv error: recv got EOF from 127.0.0.1:56512 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 14:51:48.642062 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.641193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.13.146.189:38125)
0812 14:51:48.641312 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.641314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.641544 (+ 230us) server_negotiation.cc:407] Connection header received
0812 14:51:48.641644 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.641645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.641654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.641687 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.642019 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56524: BlockingRecv error: recv got EOF from 127.0.0.1:56524 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260812 14:51:48.648087 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.647247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.13.146.189:38125)
0812 14:51:48.647283 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.647285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.647496 (+ 211us) server_negotiation.cc:407] Connection header received
0812 14:51:48.647595 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.647596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.647606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.647635 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.647958 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56540: BlockingRecv error: recv got EOF from 127.0.0.1:56540 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:51:48.658416 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.657482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.13.146.189:38125)
0812 14:51:48.657526 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.657528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.657784 (+ 256us) server_negotiation.cc:407] Connection header received
0812 14:51:48.657895 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.657896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.657907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.657937 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.658353 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56546: BlockingRecv error: recv got EOF from 127.0.0.1:56546 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:51:48.676895 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.675886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.13.146.189:38125)
0812 14:51:48.675934 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.675936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.676229 (+ 293us) server_negotiation.cc:407] Connection header received
0812 14:51:48.676365 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.676366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.676379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.676411 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.676831 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56554: BlockingRecv error: recv got EOF from 127.0.0.1:56554 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 14:51:48.711511 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.710606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.13.146.189:38125)
0812 14:51:48.710663 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.710665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.710931 (+ 266us) server_negotiation.cc:407] Connection header received
0812 14:51:48.711033 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.711034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.711046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.711076 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.711452 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56570: BlockingRecv error: recv got EOF from 127.0.0.1:56570 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:51:48.778000 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.777153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.13.146.189:38125)
0812 14:51:48.777272 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.777274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.777292 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:51:48.777371 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.777373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.777383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.777439 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.777911 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56586: BlockingRecv error: recv got EOF from 127.0.0.1:56586 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 14:51:48.907876 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:48.907156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.13.146.189:38125)
0812 14:51:48.907227 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:48.907229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:48.907376 (+ 147us) server_negotiation.cc:407] Connection header received
0812 14:51:48.907416 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:48.907417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:48.907430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:48.907483 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:48.907814 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56590: BlockingRecv error: recv got EOF from 127.0.0.1:56590 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 14:51:49.165930 24177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:51:49.165183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.13.146.189:38125)
0812 14:51:49.165267 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.165269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.165343 (+ 74us) server_negotiation.cc:407] Connection header received
0812 14:51:49.165394 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.165394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.165406 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.165461 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:51:49.165853 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56594: BlockingRecv error: recv got EOF from 127.0.0.1:56594 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:51:49.510865 24378 catalog_manager.cc:1259] Loaded cluster ID: 559dffc4cdc7424b95e3afa023f6a09f
I20260812 14:51:49.510986 24378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: loading cluster ID for follower catalog manager: success
I20260812 14:51:49.511842 24378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: acquiring CA information for follower catalog manager: success
I20260812 14:51:49.512190 24378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1ed3624697bc434f9b717f281d285958: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:51:49.564546 24455 catalog_manager.cc:1259] Loaded cluster ID: 559dffc4cdc7424b95e3afa023f6a09f
I20260812 14:51:49.564715 24455 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: loading cluster ID for follower catalog manager: success
I20260812 14:51:49.565601 24455 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: acquiring CA information for follower catalog manager: success
I20260812 14:51:49.565958 24455 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 495454a403b841d292108ecea46ed6e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:51:49.681838 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:49.679218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.13.146.189:38125)
0812 14:51:49.679345 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.679347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.679416 (+ 69us) server_negotiation.cc:407] Connection header received
0812 14:51:49.679468 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.679469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.679526 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.679617 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:49.679776 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.680355 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.681101 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.681318 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.681370 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:49.681535 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:49.681645 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:49.681704 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:49.681753 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 14:51:49.681770 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260812 14:51:49.686293 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:49.683699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.13.146.188:34297)
0812 14:51:49.683828 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.683829 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.683837 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:51:49.683875 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.683876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.683913 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.683986 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:49.684177 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.684652 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.685454 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.685739 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.685800 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:49.685928 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:49.686031 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:49.686090 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:49.686138 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:51:49.686155 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260812 14:51:49.691226 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:49.688275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.13.146.189:38125)
0812 14:51:49.688576 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.688578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.688587 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:51:49.688613 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.688614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.688626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.688677 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:49.688804 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.689567 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690438 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.690716 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690780 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:49.690921 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:49.690958 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:49.691016 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:49.691096 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:51:49.691126 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260812 14:51:49.691339 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:49.688042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.13.146.190:46679)
0812 14:51:49.688159 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.688161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.688171 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:51:49.688222 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.688223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.688280 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.688364 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:49.688485 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.689149 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690078 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.690399 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690468 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:49.690696 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:49.690835 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:49.690910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:49.691196 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 14:51:49.691226 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260812 14:51:49.691442 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:51:49.688549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.13.146.188:34297)
0812 14:51:49.688589 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:51:49.688591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:51:49.688598 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:51:49.688621 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:51:49.688622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:51:49.688631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:51:49.688680 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:51:49.688867 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.689617 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690573 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:51:49.690868 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:51:49.690947 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:51:49.691146 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:51:49.691178 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:51:49.691209 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 14:51:49.691315 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 14:51:49.691336 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:51:49.693392 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 23894
I20260812 14:51:49.712791 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24030
I20260812 14:51:49.721807 13898 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskS5QCDN/build/release/bin/kudu with pid 24181
2026-08-12T14:51:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3262 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6401 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289226 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST